Method and system for providing recommendation information related to photography

ABSTRACT

An electronic apparatus and a method by which the electronic apparatus provides recommendation information related to photography are provided. The method includes detecting, by an electronic device, a face of a subject in a preview screen viewed by a camera of the electronic device and displayed on a display of the electronic device, identifying a current composition information of the preview screen based on the detected face of the subject within the preview screen, determining a recommended photographing composition based at least in part on the identified current composition information of the preview screen and a central composition information, and providing a visual composition guide on the display of the electronic device based on the determined recommended photographing composition, the visual composition guide including a current composition indicator and a recommended photographing composition indicator.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 16/294,339, filed on Mar. 6, 2019; which is a continuationapplication of prior application Ser. No. 16/161,761, filed on Oct. 16,2018; and which was based on and claimed priority under 35 U.S.C. § 119of a Korean patent application number 10-2017-0164328, filed on Dec. 1,2017, in the Korean Intellectual Property Office, the disclosure of eachof which is incorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to artificial intelligence (AI) systems forsimulating functions of the human brain such as recognition anddecision-making by using machine learning algorithms such as deeplearning, and applications of the AI systems. More particularly, thedisclosure relates to methods and apparatuses for providingrecommendation information related to photography by using AI systems.

2. Description of Related Art

An artificial intelligence (AI) system is a computer system configuredto realize human-level intelligence and get smarter throughself-learning and making decisions spontaneously, unlike an existingrule-based smart system. The more an AI system is used, the more itsrecognition rate improves and the more accurately it understands auser's taste, and thus, the rule-based smart system is gradually beingreplaced by a deep learning-based AI system.

AI technology includes machine learning (e.g., deep learning) andelement technologies that use machine learning.

Machine learning is an algorithm technology that self-classifies andlearns characteristics of input data, and element technologies aretechnologies using a machine learning algorithm such as deep learning tosimulate functions of the human brain such as recognition anddecision-making, and include technical fields such as linguisticunderstanding, visual understanding, inference/prediction, knowledgerepresentation, and motion control.

Various fields to which AI technology is applied are as follows.Linguistic understanding is a technology for recognizing andapplying/processing human languages/characters and includes naturallanguage processing, machine translation, dialog systems, questions andanswering, and voice recognition/synthesis. Visual understanding is atechnology for recognizing and processing objects in the manner of ahuman visual system and includes object recognition, object tracking,image searching, person recognition, scene understanding, spatialunderstanding, and image enhancement. Inference/prediction is atechnology for judging information and logically inferring andpredicting the same and includes knowledge/probability-based reasoning,optimization prediction, preference-based planning, and recommendation.Knowledge representation is an automation technology for incorporatinghuman experience information into knowledge data and includes knowledgebuilding (e.g., data generation/classification), and knowledgemanagement (e.g., data utilization). Motion control is a technology forcontrolling self-driving of autonomous vehicles and the motion of robotsand includes movement control (e.g., navigation, collision avoidance, ordriving), and manipulation control (e.g., behavior control).

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providemethods and systems for providing recommendation information (e.g., atleast one recommended photographing composition, a recommendationsetting value, and at least one recommended pose) related to photographyby using a subject and surrounding environment information of thesubject.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, a method by which anelectronic apparatus provides recommendation information related tophotography is provided. The method includes identifying a subjectincluded in a preview image recognized by a first camera, obtaininginformation of the identified subject, obtaining information related tolight in surroundings of the identified subject, determining arecommended photographing composition based on the information of theidentified subject and the information related to the light in thesurroundings of the subject, and providing information about therecommended photographing composition.

In accordance with another aspect of the disclosure, an electronicapparatus is provided. The electronic apparatus includes an outputinterface for displaying a preview image including a subject recognizedby a first camera, a sensor for obtaining information related to lightin surroundings of the subject, and at least one processor configured toidentify the subject included in the preview image, obtain informationof the identified subject, determine a recommended photographingcomposition based on the information of the identified subject and theinformation related to the light in the surroundings of the subject, andprovide information about the recommended photographing composition.

In accordance with another aspect of the disclosure, a computer programproduct is provided. The computer program product includes acomputer-readable storage medium, wherein the computer-readable storagemedium includes instructions for identifying a subject included in apreview image recognized by a first camera, obtaining information of theidentified subject, obtaining information related to light insurroundings of the identified subject, determining a recommendedphotographing composition based on the information of the identifiedsubject and the information related to the light in the surroundings ofthe subject, and providing information about the recommendedphotographing composition.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a view for describing a photography system according to anembodiment of the disclosure;

FIG. 2 is a flowchart for describing a method by which an electronicapparatus provides recommendation information related to photography,according to an embodiment of the disclosure;

FIG. 3 is a diagram for describing an operation of generating arecommendation model through deep learning, according to an embodimentof the disclosure;

FIG. 4 is a view for describing photographing compositions according toan embodiment of the disclosure;

FIG. 5 is a view for describing an operation of detecting a currentphotographing composition, according to an embodiment of the disclosure;

FIG. 6 is a view for describing a recommended photographing compositionaccording to an embodiment of the disclosure;

FIG. 7 is a flowchart for describing a method of recommending orautomatically applying a photographing setting value, according to anembodiment of the disclosure;

FIG. 8 is a view for describing a photographing setting value accordingto an embodiment of the disclosure;

FIG. 9 is a flowchart for describing a method of providing informationabout a recommended photographing composition by using an image having aviewing angle greater than that of a preview image, according to anembodiment of the disclosure;

FIG. 10 is a view for describing use of dual cameras according to anembodiment of the disclosure;

FIG. 11 is a view for describing an operation of guiding photographingby using a maximum viewing angle image, according to an embodiment ofthe disclosure;

FIGS. 12 and 13 are views for describing an operation of displayinginformation for guiding a recommended photographing composition on amaximum viewing angle image, according to various embodiments of thedisclosure;

FIGS. 14 and 15 are views for describing an operation by which theelectronic apparatus displays information about a recommendedphotographing composition, according to various embodiments of thedisclosure;

FIG. 16 is a flowchart for describing a method by which the electronicapparatus interoperates with a server to provide information about arecommended photographing composition, according to an embodiment of thedisclosure;

FIG. 17 is a view for describing an operation by which the electronicapparatus executes an artificial intelligence (AI) assistantapplication, according to an embodiment of the disclosure;

FIG. 18 is a flowchart for describing a method of providing informationabout a plurality of recommended photographing compositions, accordingto an embodiment of the disclosure;

FIGS. 19, 20, and 21 are views for describing an operation by which theelectronic apparatus displays information about a plurality ofrecommended photographing compositions, according to various embodimentsof the disclosure;

FIG. 22 is a flowchart for describing a method of providing informationabout a recommended pose, according to an embodiment of the disclosure;

FIG. 23 is a view for describing an operation by which the electronicapparatus displays information about a recommended pose, according to anembodiment of the disclosure;

FIG. 24 is a view for describing an operation of providing informationabout a recommended pose by using the number of subjects and surroundingenvironment information of the subjects, according to an embodiment ofthe disclosure;

FIG. 25 is a view for describing an operation of providing informationabout a plurality of recommended poses, according to an embodiment ofthe disclosure;

FIG. 26 is a view for describing an operation of recommending an optimalface composition, according to an embodiment of the disclosure;

FIGS. 27 and 28 are block diagrams for describing the electronicapparatus according to various embodiments of the disclosure;

FIG. 29 is a block diagram of a processor according to an embodiment ofthe disclosure;

FIG. 30 is a block diagram of a data learner according to an embodimentof the disclosure;

FIG. 31 is a block diagram of a data recognizer according to anembodiment; and

FIG. 32 is a view illustrating an example where the electronic apparatusand the server interoperate to learn and recognize data, according to anembodiment of the disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skill in the artthat the following description of various embodiments of the disclosureis provided for illustration purpose only and not for the purpose oflimiting the disclosure as defined by the appended claims and theirequivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

Throughout the specification, when a part “includes” an element, it isto be understood that the part additionally includes other elementsrather than excluding other elements as long as there is no particularopposing recitation. Also, the terms such as “ . . . unit”, “module”, orthe like used in the specification indicate a unit, which processes atleast one function or motion, and the unit may be implemented byhardware or software, or by a combination of hardware and software.

Embodiments of the disclosure will be described in detail in order tofully convey the scope of the disclosure and enable one of ordinaryskill in the art to embody and practice the disclosure. The disclosuremay, however, be embodied in many different forms and should not beconstrued as being limited to the embodiments set forth herein. Also,parts in the drawings unrelated to the detailed description are omittedto ensure clarity of the disclosure. Like reference numerals in thedrawings denote like elements.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items. Expressions such as “atleast one of,” when preceding a list of elements, modify the entire listof elements and do not modify the individual elements of the list.

FIG. 1 is a view for describing a photography system according to anembodiment of the disclosure.

Referring to FIG. 1, the photography system according to an embodimentmay include an electronic apparatus 1000. According to an embodiment,the photography system may further include a server (not shown) inaddition to the electronic apparatus 1000. An embodiment where thephotography system includes the electronic apparatus 1000 and the serverwill be described below in detail with reference to FIG. 16.

According to an embodiment, the electronic apparatus 1000 may refer to adevice for obtaining at least one frame of a subject. For convenience ofexplanation, the at least one frame of the subject may be represented asan image or a photograph.

According to an embodiment, the subject may refer to an object to bephotographed. The subject may be a moving object such as a person, ananimal, an insect, or a vehicle, an immovable object such as a building,a statue, a picture, or a rock, a plant such as a tree or a flower, alandscape such as a sea, a mountain, or a sunset, or a naturalphenomenon such as a lunar eclipse or a solar eclipse.

The electronic apparatus 1000 according to an embodiment may beimplemented in any of various forms. Examples of the electronicapparatus 1000 described herein may include, but are not limited to, adigital camera, a smartphone, a laptop computer, a tablet PC, anelectronic book terminal, a digital broadcast terminal, a personaldigital assistant (PDA), a portable multimedia player (PMP), anavigation system, and an MP3 player. The electronic apparatus 1000described herein may be a wearable device that may be worn on a user.The wearable device may be at least one of an accessory-type device(e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, orcontact lenses), a head-mounted-device (HMD), a fabric orclothing-integrated device (e.g., electronic clothing), abody-attachable device (e.g., a skin pad), and a bio-implantable device(e.g., an implantable circuit). Hereinafter, for convenience ofdescription, the following will be described on the assumption that theelectronic apparatus 1000 is a digital camera or a smartphone equippedwith a camera.

According to an embodiment, a user of the electronic apparatus 1000 maytake a photograph of the subject by using the electronic apparatus 1000.In this case, the user of the electronic apparatus 1000 may determine aphotographing composition, adjust a photographing setting value, oradjust a pose of the subject in order to take a nice photograph.However, it is different for ordinary users who are not familiar withphotography to take a nice photograph like professional photographers.

Accordingly, according to an embodiment, the electronic apparatus 1000may provide recommendation information related to photography so thatordinary users may take a nice photograph. For example, the electronicapparatus 1000 may provide, to the user, at least one from among, butnot limited to, recommendation information related to a photographingcomposition, recommendation information related to a photographingsetting value, and recommendation information related to a pose of thesubject.

An operation by which the electronic apparatus 1000 providesrecommendation information (e.g., recommendation of a photographingcomposition) related to photography by using a subject and surroundingenvironment information of the subject will now be described in detailwith reference to FIG. 2.

FIG. 2 is a flowchart for describing a method by which the electronicapparatus 1000 provides recommendation information related tophotography according to an embodiment of the disclosure.

In operation S210, the electronic apparatus 1000 may identify a subjectincluded a preview image recognized through a first camera. The previewimage may be an image that may be previewed by a user through aviewfinder or a screen before photographing.

According to an embodiment, the electronic apparatus 1000 may identifythe subject included in the preview image by analyzing the previewimage. In this case, according to an embodiment, the electronicapparatus 1000 may identify the subject included in the preview image byusing a learning network model of an artificial intelligence (AI)system.

According to an embodiment, the electronic apparatus 1000 may determinea type of the subject included in the preview image. For example, theelectronic apparatus 1000 may determine whether the subject is a person,an animal, a landscape, a natural object, or a food.

According to an embodiment, the electronic apparatus 1000 may determinethe number of subjects. For example, when the subject is a person, theelectronic apparatus 1000 may determine whether the person included inthe preview image is one person or a group. Also, when the subject is agroup, the electronic apparatus 1000 may determine how many people areincluded in the group.

According to an embodiment, the electronic apparatus 1000 may determinea main subject by using focus information or information about a ratioof spaces occupied by subjects. For example, when a field, a person, anda tree are included in the preview image, the electronic apparatus 1000may determine the person as a main subject or may determine the tree asa main subject according to a ratio of spaces occupied by the person andthe tree on the preview image. Also, the electronic apparatus 1000 maydetermine one focused person from among people included in the previewimage as a main subject.

According to an embodiment, the electronic apparatus 1000 may determinea main subject according to a user input. For example, the electronicapparatus 1000 may determine a subject selected by the user as an objectof interest as a main subject.

According to an embodiment, the electronic apparatus 1000 may identifythe subject included in the preview image through a server. For example,when transmitting information about the preview image (e.g., the previewimage or feature information extracted from the preview image) to theserver, the electronic apparatus 1000 may request the server to identifythe subject included in the preview image. In this case, the server mayidentify the subject included in the preview image by analyzing theinformation about the preview image by using the learning network model.The server may transmit identification information of the subject (e.g.,a type of the subject, the number of subjects, and a main subject) tothe electronic apparatus 1000.

In operation S220, the electronic apparatus 1000 may obtain surroundingenvironment information of the identified subject.

According to an embodiment, the surrounding environment information ofthe subject may include at least one of information related to light,information related to a place, information related to a time, andinformation related to weather.

According to an embodiment, the electronic apparatus 1000 may obtaininformation related to light in surroundings of the subject as thesurrounding environment information of the subject. For example, theinformation related to the light in the surroundings of the subject mayinclude, but is not limited to, a type of light (e.g., natural light,direct light, diffused light, artificial light, front light, side light,or backlight), scattering of light, a direction of light, an intensityof light, a position of the sun, an illuminance, auxiliary light(strobe) (e.g., an internal strobe of a camera, an external strobe, aring strobe, or a reflective plate), position information (e.g., globalpositioning system (GPS) coordinates, a region, or a country), whether alocation is indoors (e.g., a general home, an office, a banquet hall, aperformance hall, or an exhibition) or outdoors (e.g., a forest, abeach, a sea, or a firework), a time (e.g., midday, a sunrise and asunset, backlight, right after a sunset, or a late night), weather(e.g., a rainy day, a snowy day, or a winter snow scene (after snowstops), and a season.

According to an embodiment, the electronic apparatus 1000 may obtain theinformation related to the light in the surroundings of the subject byusing a current position of the electronic apparatus 1000 and a currentposition of the sun. For example, the electronic apparatus 1000 mayrecognize a current position by using a position sensor (e.g., a GPS)and may recognize a position of the sun (e.g., a direction or analtitude) by using current time information. In this case, the positionof the sun may be an absolute position of the sun, or may be a relativeposition from the electronic apparatus 1000 or the subject. For example,the electronic apparatus 1000 may recognize that the sun is located at alow altitude in a northeast direction from the subject.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information (e.g., the information related tothe light in the surroundings of the identified subject) of the subjectby analyzing the preview image. In this case, according to anembodiment, the electronic apparatus 1000 may obtain the surroundingenvironment information of the subject from the preview image by usingthe learning network model (e.g., an AI model) of the Al system. Forexample, the electronic apparatus 1000 may determine whether it isdaytime or nighttime or may determine a direction of light, an intensityof light, or a position of the sun by analyzing light and shade, ashadow, or the sun included in the preview image. Also, the electronicapparatus 1000 may determine whether a season is spring, summer, autumn,or winter or may determine whether a location is indoors or outdoors byanalyzing the subject included in the preview image. For example, theelectronic apparatus 1000 may determine that a season is summer when thesubject wears a short-sleeve shirt and may determine that a season iswinter when the subject stands on snow.

According to an embodiment, the electronic apparatus 1000 may obtaininformation related to light emitted from the subject in the previewimage by analyzing the preview image. For example, when a night scene isincluded in the preview image, the electronic apparatus 1000 mayrecognize, but is not limited to, light emitted by a lighting device(e.g., a street lamp or a car light) included in the preview image,light emitted by a planet (e.g., the moon, a star, the Venus, or themilky way), light emitted by a campfire, or light emitted by afirecracker. The electronic apparatus 1000 may obtain information aboutthe impression of a color, an intensity, and an illuminance of lightemitted by the subject in the preview image.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information (e.g., the information related tothe light in the surroundings of the identified subject) of the subjectby using at least one sensor. For example, the electronic apparatus 1000may obtain information such as a current position of the subject orwhether the subject exists indoors or outdoors by using a positionsensor.

According to an embodiment, the electronic apparatus 1000 may obtain anilluminance value of the surroundings of the subject. Also, theelectronic apparatus 1000 may determine whether the surroundings of thesubject are currently in daytime or nighttime by using an illuminancesensor.

According to an embodiment, the electronic apparatus 1000 may obtaintemperature information of the surroundings of the subject by using atemperature sensor, and may obtain humidity information of thesurroundings of the subject by using a humidity sensor.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information by using a plurality of imagesensors (or a plurality of cameras). For example, the electronicapparatus 1000 may obtain a first image (e.g., a maximum viewing angleimage) having a viewing angle greater than that of the preview image byusing a second camera different from the first camera that obtains thepreview image. The electronic apparatus 1000 may obtain the surroundingenvironment information (e.g., the information related to the light inthe surroundings of the subject) of the subject by analyzing theobtained first image (e.g., the maximum viewing angle image). Since themaximum viewing angle image includes more surrounding information thanthe preview image, information obtained by analyzing the maximum viewingangle image may be more precise than information obtained by analyzingthe preview image. An operation by which the electronic apparatus 1000obtains the surrounding environment information by using dual cameraswill be described below in detail with reference to FIG. 9.

In operation S230, the electronic apparatus 1000 may determine arecommended photographing composition by using information of theidentified subject and the surrounding environment information of thesubject. The information of the identified subject may include at leastone of, but not limited to, a type of the subject, the number ofsubjects, a main subject, a pose of the subject, and a color of thesubject.

According to an embodiment, the recommended photographing compositionmay refer to a photographing composition, a photographing area, or aphotographing angle recommended by the electronic apparatus 1000 to theuser of the electronic apparatus 1000.

The photographing composition may refer to an arrangement of subjects(or a background) in a frame. For example, referring to FIG. 4, thephotographing composition may include, but is not limited to, a goldendivision composition/rule-of-thirds composition 401, a perfect diagonalcomposition 402, a diamond-shaped composition 403, a checkerboardcomposition 404, a parallel horizontal composition 405, a verticalcomposition 406, a horizontal one-third composition 407, a horizontaland vertical composition 408, an inverted triangular composition 409, aspiral composition 410, a circular composition 411, a lightning-shapedcomposition 412, a U-shaped composition 413, a central composition 414,a C-shaped composition 415, an S-shaped composition 416, a fan-shapedcomposition 417, a diagonal composition 418, a parallel diagonalcomposition 419, a triangular composition 420, a horizontal composition421, a cross-shaped composition 422, an arched composition 423, anX-shaped diagonal composition 424, a radial composition 425, and abilateral symmetrical composition 426.

According to an embodiment, the electronic apparatus 1000 may determinethe recommended photographing composition by using the AI model. Forexample, the electronic apparatus 1000 may determine the recommendedphotographing composition by using the learning network model (e.g., theAI model) trained based on photographs taken by professionals. Thelearning network model (e.g., the AI model) trained based on thephotographs taken by the professionals will be described below in detailwith reference to FIG. 3.

According to an embodiment, the electronic apparatus 1000 may determinethe recommended photographing composition according to a type of theidentified subject and the number of subjects. For example, when thesubject is a food and several dishes are located, the electronicapparatus 1000 may determine a first photographing composition suitablefor an image where several foods are arranged as a recommendedphotographing composition, and when the subject is one person, theelectronic apparatus 1000 may determine a second photographingcomposition as a recommended photographing composition.

According to an embodiment, the electronic apparatus 1000 may determinethe recommended photographing composition by using the information ofthe identified subject and the information related to the light in thesurroundings of the subject.

For example, when foods that are subjects are located outdoors, theelectronic apparatus may determine a third photographing composition asa recommended photographing composition in consideration of a directionof light, and when foods that are subjects are located indoors, theelectronic apparatus 1000 may determine a fourth photographingcomposition as a recommended photographing composition.

According to an embodiment, the electronic apparatus 1000 may determinea composition where the sun is not backlight as the recommendedphotographing composition, in consideration of a position of the sun.Alternatively, when the sun is backlight, the electronic apparatus 1000may determine a composition where the subject looks best as therecommended photographing composition.

Also, the electronic apparatus 1000 may determine the recommendedphotographing composition for long exposure photography by usinginformation emitted by the subject or information related to lightemitted by an object around the subject. For example, the electronicapparatus 1000 may determine an S-shaped composition where the subjectis located on the right as the recommended photographing composition forlong exposure photography, in consideration of a direction, anintensity, and the impression of a color of light of a street lamp.Alternatively, the electronic apparatus 1000 may determine an exposurewhere the land occupies ⅓ and the sky occupies ⅔ on the preview image asthe recommended photographing composition for long exposure photography,in consideration of a position and a brightness of the moon or a star.

According to an embodiment, as a result obtained by checking theinformation related the light in the surroundings of the subject, whenit is nighttime and a location is outdoors, a good photograph may beobtained by using long exposure. Accordingly, the electronic apparatus1000 may recommend a photographing setting value for long exposurephotography. A method by which the electronic apparatus 1000 recommendsthe photographing setting value will be described below in detail withreference to FIG. 7.

According to an embodiment, the electronic apparatus 1000 may determinethe recommended photographing composition by using photographs taken byprofessionals. For example, when the subject is a particular building(e.g., the Taj Mahal shown in FIG. 11) and the sun is located at theleft of the building, the electronic apparatus 1000 may determine afifth photographing composition as the recommended photographingcomposition, based on photographs taken when the sun is located at theleft of the building from among a plurality of photographs taken byprofessionals who have photographed the building. Alternatively, whenthe subject is a particular mountain (e.g., Mt. Everest) and themountain is foggy as a result obtained by analyzing the surroundingenvironment information, the electronic apparatus 1000 may determine asixth photographing composition as the recommended photographingcomposition, based on photographs taken on a foggy morning from amongphotographs taken by professionals who photographed the mountain.According to an embodiment, when the subject is a particular beach(e.g., Waikiki beach) and there is a sunset as a result obtained byanalyzing the surrounding environment information, the electronicapparatus 1000 may determine a photographing composition of aprofessional who well represents the sunset of the beach as therecommended photographing composition.

According to an embodiment, when the identified subject is a tallbuilding and a characteristic object does not exist in the surroundingsas an analysis result based on the maximum viewing angle image, theelectronic apparatus 1000 may determine a vertical mode as therecommended photographing composition. Also, when the subject is severalpeople and the people stand on a wide lawn outdoors as an analysisresult based on the maximum viewing angle image, the electronicapparatus 1000 may determine a horizontal mode as the recommendedphotographing composition.

According to an embodiment, when a main subject is a person and there isa projecting subject such as a tree or a pillar on a background as aresult obtained by analyzing the preview image, the electronic apparatus1000 may determine a composition where the tree or the pillar does notoverlap the person as the recommended photographing composition. Also,when a main subject is a person and a beach is a background as a resultobtained by analyzing the preview image, the electronic apparatus 1000may determine a composition where the horizon of the sea does not passthrough the person's eyes or neck as the recommended photographingcomposition.

According to an embodiment, when a main subject is a person, theelectronic apparatus 1000 may determine the recommended photographingcomposition in consideration of the person's gaze. For example, when thegaze of the person who is the subject is on the right, the electronicapparatus 1000 may determine a composition where there is a space on theright as the recommended photographing composition, and when the gaze ofthe person is on the left, the electronic apparatus 1000 may determine acomposition where there is a space on the left as the recommendedphotographing composition.

According to an embodiment, when a person is photographed outdoors, theelectronic apparatus 1000 may determine a composition where a portraitphotograph may be taken in harmony with a background as the recommendedphotographing composition. For example, the electronic apparatus 1000may determine a rule-of-thirds composition as the recommendedphotographing composition. The rule-of-thirds composition refers to acomposition where a frame is divided by two virtual horizontal lines andtwo virtual vertical lines and the subject is placed along virtualintersections. A stable photograph may be obtained when photographing inthe rule-of-thirds composition.

According to an embodiment, the electronic apparatus 1000 may determinethe recommended photographing composition in consideration of aphotographing area and a photographing angle. For example, when only aportion over a person's neck is included in the preview image, theelectronic apparatus 1000 may determine a composition where a portionover the person's chest may be included in the preview image inconsideration of the surrounding environment information as therecommended photographing composition. In this case, the recommendedphotographing composition may include information for reducing a zoommagnification from a current magnification (zoom out) and reducing anangle of a camera.

According to an embodiment, the electronic apparatus 1000 may transmitthe information about the preview image or the surrounding environmentinformation of the subject to the server and may request the server torecommend a photographing composition. In this case, the server maydetermine the recommended photographing composition by using theinformation about the preview image or the surrounding environmentinformation of the subject, and may transmit information about thedetermined recommended photographing composition to the electronicapparatus 1000. An operation by which the electronic apparatus 1000interoperates with the server to determine the recommended photographingcomposition will be described below in detail with reference to FIG. 16.

In operation S240, the electronic apparatus 1000 may provide theinformation about the determined recommended photographing composition.

According to an embodiment, the electronic apparatus 1000 may determinea current photographing composition from the preview image. For example,the electronic apparatus 1000 may determine the current photographingcomposition according to a shape and a position of the subject.Alternatively, the electronic apparatus 1000 may detect lines on thepreview image and may determine the current photographing composition byusing the detected lines.

According to an embodiment, the electronic apparatus 1000 may comparethe recommended photographing composition with the current photographingcomposition. When a similarity between the recommended photographingcomposition and the current photographing composition is less than athreshold value (e.g., 97%) as a result of the comparison, theelectronic apparatus 1000 may provide information about the recommendedphotographing composition. According to an embodiment, when a similaritybetween the recommended photographing composition and the currentphotographing composition is equal to or greater than the thresholdvalue (e.g., 97%), the electronic apparatus 1000 may not provide theinformation about the recommended photographing composition.

According to an embodiment, the electronic apparatus 1000 may providethe information about the recommended photographing composition byoutputting information for guiding the recommended photographingcomposition. In this case, the electronic apparatus 1000 may output theinformation for guiding the recommended photographing composition byusing at least one of, but not limited to, a video signal, an audiosignal, and a vibration signal. For example, the electronic apparatus1000 may display text or an icon for guiding the recommendedphotographing composition on the preview image. The electronic apparatus1000 may output a voice (e.g., “Slightly move the camera down andright”) for guiding the recommended photographing composition.

According to an embodiment, the electronic apparatus 1000 may determinea plurality of recommended photographing compositions by using theinformation of the subject and the surrounding environment informationof the subject. In this case, the electronic apparatus 1000 may providethumbnail images respectively corresponding to the plurality ofrecommended photographing compositions. Also, the electronic apparatus1000 may provide preview images respectively corresponding to theplurality of recommended photographing compositions. An operation bywhich the electronic apparatus 1000 provides information about theplurality of recommended photographing compositions will be describedbelow in detail with reference to FIG. 18.

FIG. 3 is a diagram for describing an operation of generating arecommendation model through deep learning according to an embodiment ofthe disclosure.

Referring to FIG. 3, according to an embodiment, an AI processorincluded in a server or the electronic apparatus 1000 may generate arecommendation model 300 that recommends a photographing composition orderives a photographing setting value by training an artificial neuralnetwork. When the artificial neural network is ‘trained’, it may meanthat a mathematical model that allows connections of neuronsconstituting the artificial neural network to make an optimaldetermination by appropriately changing weights based on data iscreated.

According to an embodiment, the AI processor may obtain photographimages 301, photograph metadata 302, and surrounding environmentinformation 303, and may generate the recommendation model 300 by usingthe photograph images 301, the photograph metadata 302, and thesurrounding environment information 303.

According to an embodiment, the photograph images 301 obtained by the AIprocessor may be many photograph images taken by professionals, and mayinclude information about a subject and a photographing composition.According to an embodiment, the photograph image 301 may be a stillimage, or a plurality of frames included in a video.

According to an embodiment, the photograph metadata 302 may include, butis not limited to, a camera maker, a camera model, a lens type, an imageeditor (software), a date/time when a photograph is edited, anexchangeable image file format (Exif) version, a shooting date/time, anactual size of a photograph, an exposure time (or a shutter speed), anexposure program, a lens focal length, an F-number of a stop, whether aflash is used, and a white balance. According to an embodiment, thephotograph metadata 302 may be Exif data.

According to an embodiment, the surrounding environment information 303may refer to information related to a surrounding environment of thesubject included in the photograph image 301. For example, thesurrounding environment information 303 may include, but is not limitedto, a type of light (e.g., natural light, direct light, diffused light,artificial light, frontal light, side light, or backlight), scatteringof light, a direction of light, an intensity of light, a position of thesun, an illuminance, auxiliary light (strobe) (e.g., an internal strobeof a camera, an external strobe, a ring strobe, or a reflective plate),position information (e.g., GPS coordinates, a region, or a country),whether a location is indoors (e.g., a general home, an office, abanquet hall, a performance hall, or an exhibition) or outdoors (e.g., aforest, a beach, a sea, or a firework), a time (e.g., midday, a sunriseand a sunset, backlight, right after a sunset, or a late night), weather(e.g., a rainy day, a snowy day, or a winter snow scene (after snowstops), and a season.

According to an embodiment, the AI processor may model a recommendedphotographing composition or a recommendation photographing settingvalue by identifying the photographing composition and the subject fromthe photograph images 301 of the professionals and matching theidentified subject and the identified photographing composition tocorresponding metadata (photographing setting value) and correspondingsurrounding environment information. According to an embodiment, as thephotograph images 301 of the professionals, the corresponding metadata,and the corresponding surrounding environment information collected bythe AI processor increase, the recommendation model 300 that derives anoptimal photographing composition and an optimal photographing settingvalue may be modified.

According to an embodiment, the AI processor may obtain personalizedlearning data 304. The personalized learning data 304 may be data aboutan individual's photographic taste. For example, the personalizedlearning data 304 may include, but is not limited to, data about aphotographing composition (or a photographing setting value) preferredby the individual, data about a photographing composition (or aphotographing setting value) not selected by the individual, data aboutphotographing compositions (or photographing setting values) ofphotographs deleted by the individual, data about whether a recommendedphotographing composition (or a photographing setting value) is applied,and data about a photograph finally taken by the individual.

According to an embodiment, the AI processor may generate therecommendation model 300 for each individual by using the personalizedlearning data 304 in addition to the photograph image 301, thephotograph metadata 302, and the surrounding environment information303. For example, the AI processor may recommend a first photographingcomposition to a first user and a second photographing composition to asecond user under the same surrounding environment for the same subject,by using the personalized learning data 304.

An embodiment where the electronic apparatus 1000 detects a currentphotographing composition from a preview image and provides informationabout a recommended photographing composition by using therecommendation model 300 will now be described with reference to FIGS. 5and 6.

FIG. 5 is a view for describing an operation of detecting a currentphotographing composition according to an embodiment of the disclosure.

Referring to FIG. 5, the electronic apparatus 1000 may determine acurrent photographing composition (for convenience of explanation,referred to as a ‘current composition’) according to a shape and aposition of a subject. For example, the electronic apparatus 1000 maydetect points or lines in relation to the subject on a preview image.The electronic apparatus 1000 may determine a current photographingcomposition by using the detected points or lines.

Referring to 510 of FIG. 5, when a user is to photograph the Taj Mahaland surroundings of the Taj Mahal by using the electronic apparatus1000, the Taj Mahal and a surrounding environment may be included in thepreview image. In this case, the electronic apparatus 1000 may detect aline 1 that connects a pillar located at the right of the Taj Mahal, aline 2 that connects a pillar located at the left of the Taj Mahal,lines 3 that connect the vertex of the Taj Mahal and the two pillars,and a horizon 4, on the preview image. The electronic apparatus 1000 maydetermine the Taj Mahal on the preview image as a main subject, and maydetermine that a current photographing composition is a triangularcomposition 420 based on the lines 3 that connect the vertex of the TajMahal and the two pillars.

Referring to 520 of FIG. 5, when it is determined that the currentphotographing composition is the triangular composition 420, theelectronic apparatus 1000 may determine a virtual central line 500 thathalves a triangle connecting the vertex of the Taj Mahal and the twopillars. The electronic apparatus 1000 may determine a composition wherethe virtual central line 500 is located at the center of the previewimage as a recommended photographing composition (for convenience ofexplanation, referred to as a ‘recommendation composition’). Arecommendation photographing compassion will now be described in detailwith reference to FIG. 6.

FIG. 6 is a view for describing a recommended photographing compositionaccording to an embodiment of the disclosure.

Referring to 600-1 of FIG. 6, the electronic apparatus 1000 may detect acurrent photographing composition on a preview image 600 including theTaj Mahal. The electronic apparatus 1000 may identify a subject includedin the preview image 600 and may determine the current photographingcomposition based on the identified subject.

According to an embodiment, the electronic apparatus 1000 may determinea recommended photographing composition by using the recommendationmodel 300 trained based on a plurality of photographs. The electronicapparatus 1000 may determine the recommended photographing compositionin consideration of information of the subject (e.g., the Taj Mahal) andsurrounding environment information (e.g., a height of the sun, aseason, weather, or whether a location is outdoors). For example, sincethe subject is the Taj Mahal, the electronic apparatus 1000 maydetermine an entire composition as a triangular composition and maydetermine a composition where a central axis 601 of the currentphotographing composition moves to a central axis 602 of the previewimage 600 as the recommended photographing composition.

According to an embodiment, the electronic apparatus 1000 may provideinformation for guiding the recommended photographing composition bydisplaying a line 610 indicating the current photographing compositionand a line 620 indicating the recommended photographing composition onthe preview image 600. A user may compare the current photographingcomposition with the recommended photographing composition, and then mayslowly move a camera leftward to take a photograph of the recommendedphotographing composition. In this case, the line 610 indicating thecurrent composition on the preview image 600 may approach the line 620indicating the recommended photographing composition.

Referring to 600-2 of FIG. 6, when the user slowly moves the cameraleftward and the line 610 indicating the current photographingcomposition and the line 620 indicating the recommended photographingcomposition overlap each other (e.g., as shown in the preview image630), the user may select a photographing button. In this case, theelectronic apparatus 1000 may obtain a photograph image of the Taj Mahalaccording to the recommended photographing composition in response to auser input that selects the photographing button.

According to an embodiment, when the line 610 indicating the currentphotographing composition and the line 620 indicating the recommendedphotographing composition overlap each other, the electronic apparatus1000 may automatically perform photographing.

Although lines indicating a current photographing composition and arecommended photographing composition are displayed in order for theelectronic apparatus 1000 to guide the recommended photographingcomposition in FIG. 6, the disclosure is not limited thereto. Forexample, indicators indicating the current photographing composition andthe recommended photographing composition may be represented as any ofvarious shapes other than lines. Also, the electronic apparatus 1000 mayguide the recommended photographing composition by using a voice signal.

FIG. 7 is a flowchart for describing a method of recommending orautomatically applying a photographing setting value according to anembodiment of the disclosure.

In operation S710, the electronic apparatus 1000 may identify a subjectincluded in a preview image. For example, the electronic apparatus 1000may identify the subject included in the preview image by analyzing thepreview image. In this case, according to an embodiment, the electronicapparatus 1000 may identify the subject included in the preview image byusing a learning network model of an AI system. Operation S710corresponds to operation S210 of FIG. 2, and thus a detailed explanationthereof will not be given.

In operation S720, the electronic apparatus 1000 may obtain surroundingenvironment information of the subject.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information of the subject by analyzing thepreview image. In this case, according to an embodiment, the electronicapparatus 1000 may obtain the surrounding environment information of thesubject from the preview image by using the learning network model ofthe AI system.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information of the subject by using at least onesensor. For example, the electronic apparatus 1000 may obtaininformation about a current position of the subject or whether thesubject is located indoors or outdoors by using a position sensor. Theelectronic apparatus 1000 may obtain an illuminance value ofsurroundings of the subject by using an illuminance sensor. Also, theelectronic apparatus 1000 may determine whether the surroundings of thesubject are currently in daytime or nighttime by using the illuminancesensor. The electronic apparatus 1000 may obtain temperature informationof the surroundings of the subject by using a temperature sensor, andmay obtain humidity information of the surroundings of the subject byusing a humidity sensor. The electronic apparatus 1000 may obtain thesurrounding environment information by using a plurality of imagesensors (or a plurality of cameras).

Operation S720 corresponds to operation S220 of FIG. 2, and thus adetailed explanation thereof will not be given.

In operation S730, the electronic apparatus 1000 may recommend orautomatically apply a photographing setting value based on informationof the subject and the surrounding environment information of thesubject. According to an embodiment, the photographing setting value mayinclude, but is not limited to, an F-number of a stop, a shutter speed,an ISO sensitivity, a white balance, and an exposure value.

FIG. 8 is a view for describing a photographing setting value accordingto an embodiment of the disclosure.

Referring to FIG. 8, a stop 810 refers to a size of an aperture of alens through which light passes. As the stop 810 is closed (right) toincrease a depth, a photograph where a near portion and a far portionare focused is output, and as the stop 810 is opened (left) to reduce adepth, a photograph where a subject and a background are separated fromeach other, referred to as out-focusing, is output. As a shutter speed820 increases (left), a photograph where a fast moving object appearsfrozen is output whereas as the shutter speed 820 decreases (right), ablurred photograph is output. As an ISO sensitivity 830 decreases(left), a photograph with small noise is output. As the ISO sensitivity830 increases (right), noise increases and a photograph with no shakemay be taken even in a dark environment.

As the ISO sensitivity 830 decreases (left), a contrast increases. Incontrast, as the ISO sensitivity 830 increases, a soft photograph byreducing a contrast is taken. Film grains when the ISO sensitivity 830is low are thin and lead to a clear photograph. Film grains when the ISOsensitivity 830 is high are thick and lead to a rough photograph.

According to an embodiment, the electronic apparatus 1000 may determinea recommendation photographing setting value based on information of asubject and surrounding environment information of the subject. Forexample, when the subject is photographed outdoors where the sun isshining brightly in daytime, the electronic apparatus 1000 may determinea shutter speed as 1/4000 seconds. When the subject is photographedindoors, the electronic apparatus 1000 may determine a shutter speed as1/60 seconds. When a star is photographed on a dark night, theelectronic apparatus 1000 may determine a shutter speed as 10 seconds ormore. According to an embodiment, when the subject is photographedindoors, the electronic apparatus 1000 may determine a photographingsetting value according to a color of the subject. For example, when thesubject is a black device, the electronic apparatus 1000 may determinean F-number of a stop as 4.0 and may determine a shutter speed as 1/30seconds. In contrast, when the subject is a white device, the electronicapparatus 1000 may determine an F-number of the stop as 4.0 like theblack device, and may determine a shutter speed as 1/100 seconds.

According to an embodiment, the electronic apparatus 1000 may recommenda plurality of photographing setting value sets based on the informationof the subject and the surrounding environment information of thesubject. For example, the electronic apparatus 1000 may recommend afirst photographing setting value set (e.g., ISO: 12800, stop: 1.4, andshutter speed: 1), a second photographing setting value set (e.g., ISO:6400, stop: 2.8, and shutter speed: ½), a third photographing settingvalue set (e.g., ISO: 3200, stop: 5.6, and shutter speed: ¼), a fourthphotographing setting value set (e.g., ISO: 1600, stop: 8, and shutterspeed: ⅛), and a fifth photographing setting value set (e.g., ISO: 400,stop: 16, and shutter speed: 1/60).

According to an embodiment, the electronic apparatus 1000 may receive auser input that selects one from among the plurality of photographingsetting value sets. The electronic apparatus 1000 may apply aphotographing setting value set selected by a user to a photographingsystem.

According to an embodiment, the electronic apparatus 1000 may recommenda plurality of photographing setting value sets in which values of thestop 810, the shutter speed 820, and the ISO sensitivity 830 aredifferent but the amounts of light are the same by combinations. Forexample, the electronic apparatus 1000 may recommend a sixthphotographing setting value set (e.g., ISO: 200, stop: 2.8, and shutterspeed: ½) and a seventh photographing setting value set (e.g., ISO: 400,stop: 8, and shutter speed: 1) according to the information of thesubject and the surrounding environment information of the subject. Thesixth photographing setting value set and the seventh photographingsetting value set theoretically have the same amount of light. However,since values of the stop 810, the shutter speed 820, and the ISOsensitivity 830 are different from each other, the sixth photographingsetting value set may more efficiently photograph a moving subject andmay have a lower depth than the seventh photographing setting value set.In contrast, the seventh photographing setting value set may have morenoise and a higher depth, and may be more affected by hand shake due toa low shutter speed than the sixth photographing setting value set. Theuser may compare the sixth photographing setting value set with theseventh photographing setting value set, and may select one of the sixthphotographing setting value set and the seventh photographing settingvalue set according to a type of a desired photograph.

According to an embodiment, when a photographing composition is selectedthrough a movement of a camera, the electronic apparatus 1000 may take aplurality of photographs by applying a plurality of photographingconditions and may provide the plurality of photographs. For example,the electronic apparatus 1000 may take a plurality of photographs byapplying the first photographing setting value set (e.g., ISO: 12800,stop: 1.4, and shutter speed: 1), the second photographing setting valueset (e.g., ISO: 6400, stop: 2.8, and shutter speed: ½), the thirdphotographing setting value set (e.g., ISO: 3200, stop: 5.6, and shutterspeed: ¼), the fourth photographing setting value set (e.g., ISO: 1600,stop: 8, and shutter speed: ⅛), and the fifth photographing settingvalue set (e.g., ISO: 400, stop: 16, and shutter speed: 1/60), and mayprovide the plurality of photographs to the user.

The user may select at least one of the plurality of photographs, andmay store or delete the selected photograph. In this case, an AIprocessor of the electronic apparatus 1000 (or an AI processor of aserver) may refine the recommendation model 300 by learningphotographing setting value information corresponding to photographingsetting value information corresponding to the stored photograph orphotographing setting value information corresponding to the deletedphotograph.

FIG. 9 is a flowchart for describing a method of providing informationabout a recommended photographing composition by using an image having aviewing angle greater than that of a preview image according to anembodiment of the disclosure.

In operation S910, the electronic apparatus 1000 may obtain a previewimage by using a first camera, and may obtain an image having a viewingangle greater than that of the preview image by using a second camera.

FIG. 10 is a view for describing use of dual cameras according to anembodiment of the disclosure.

According to an embodiment, the electronic apparatus 1000 may includedual cameras. For example, referring to 1010 of FIG. 10, a first camera1001 may be a general camera and a second camera 1002 may be a telephotocamera. In this case, the electronic apparatus 1000 may obtain thepreview image by using a standard lens of the general camera, and mayobtain the image having a viewing angle greater than that of the previewimage by zooming out a telephoto lens of the telephoto camera.

The standard lens is a lens reproducing a viewing angle of a camera thatis similar to a viewing angle of a person. A lens having a focal lengthof 50 mm based on a 35 mm film camera is referred to as the standardlens. Since a viewing angle of the standard lens is generally the sameas a viewing angle of a person, a natural photograph may be taken byusing the standard lens.

The telephoto lens may refer to a lens having a focal length greaterthan that of the standard lens. Since the telephoto lens has a focallength greater than that of the standard lens, a viewing angle may beless than that of the standard lens. Accordingly, the electronicapparatus 1000 may obtain the image having a viewing angle greater thanthat of the preview image obtained by the standard lens, by zooming outthe telephoto lens.

Referring to 1020 of FIG. 10, the first camera 1001 may be a generalcamera and the second camera 1002 may be a wide-angle camera. In thiscase, the electronic apparatus 1000 may obtain the preview image byusing the standard lens of the general camera, and may obtain the imagehaving a viewing angle greater than that of the preview image by using awide-angle lens of the wide-angle camera. The wide-angle lens may referto a lens having a focal length less than that of the standard lens.Since the wide-angle lens has a focal length less than that of thestandard lens, a viewing angle may be greater than that of the standardlens. Accordingly, the electronic apparatus 1000 may obtain the imagehaving a viewing angle greater than that of the preview image obtainedby the standard lens by using the wide-angle lens.

Referring to 1030 of FIG. 10, the first camera 1001 and the secondcamera 1002 may be general cameras. In this case, the electronicapparatus 1000 may obtain the preview image by using the standard lensof the first camera 1001, and may obtain a panorama image having aviewing angle greater than that of the preview image by stitching imagesobtained by the first camera 1001 and the second camera 1002.

According to an embodiment, the image having a viewing angle greaterthan that of the preview image may be an image having a maximum viewingangle (hereinafter, referred to as a maximum viewing angle image) thatmay be obtained by the electronic apparatus 1000. For convenience ofexplanation, the following will be described on the assumption that theimage having a viewing angle greater than that of the preview image is amaximum viewing angle image.

In operation S920, the electronic apparatus 1000 may obtain surroundingenvironment information of a subject by using the image having a viewingangle greater than that of the preview image.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information of the subject by analyzing themaximum viewing angle image. In this case, according to an embodiment,the electronic apparatus 1000 may obtain the surrounding environmentinformation of the subject from the maximum viewing angle image by usinga learning network model of an AI system.

For example, the electronic apparatus 1000 may determine whether it isdaytime or nighttime, a direction of light or an intensity of light, anda position of the sun by analyzing light and shade, a shadow, and thesun included in the maximum viewing angle image. Also, the electronicapparatus 1000 may determine whether a season is spring, summer, autumn,or winter or may determine whether a location is indoors or outdoors byanalyzing the subject included in the maximum viewing angle image. Forexample, when the electronic apparatus 1000 may determine that a seasonis summer when the subject wears a short-sleeve shirt and may determinethat a season is winter when the subject stands on snow. The electronicapparatus 1000 may determine an entire color temperature by analyzingthe maximum viewing angle image.

In operation S930, the electronic apparatus 1000 may provide informationabout a recommended photographing composition by using information ofthe subject and the surrounding environment information.

According to an embodiment, the electronic apparatus 1000 may determinea current photographing composition by using the preview image. Theelectronic apparatus 1000 may determine the recommended photographingcomposition based on the information of the subject and the surroundingenvironment information of the subject. In this case, when a similaritybetween the current photographing composition and the recommendedphotographing composition is less than a threshold value, the electronicapparatus 1000 may provide the information about the recommendedphotographing composition.

According to an embodiment, the electronic apparatus 1000 may providethe information about the recommended photographing composition byoutputting information for guiding the recommended photographingcomposition. In this case, the electronic apparatus 1000 may output theinformation for guiding the recommended photographing composition byusing at least one of, but not limited to, a video signal, an audiosignal, and a vibration signal. For example, the electronic apparatus1000 may output a voice for guiding the recommended photographingcomposition. Also, the electronic apparatus 1000 may display a graphicindicator for guiding the recommended photographing composition on theimage having a viewing angle greater than that of the preview image(e.g., the maximum viewing angle image).

Operation S930 corresponds to operations S230 and S240 of FIG. 2, andthus a detailed explanation thereof will not be given. An operation bywhich the electronic apparatus 1000 provides the information for guidingthe recommended photographing composition by using the maximum viewingangle image will now be described in detail with reference to FIGS. 11through 13.

FIG. 11 is a view for describing an operation of guiding photographingby using a maximum viewing angle image according to an embodiment of thedisclosure.

Referring to FIG. 11, the electronic apparatus 1000 may obtain a previewimage by using a standard lens. In this case, a current photographingarea 1101 shown in the preview image may be a right area of the TajMahal.

The electronic apparatus 1000 may obtain a maximum viewing angle image1102 by using a wide-angle lens, a telephoto lens, or two standardlenses. The maximum viewing angle image 1102 may include an image ofsurroundings of the current photographing area 1101.

The electronic apparatus 1000 may determine a recommended photographingcomposition by analyzing the maximum viewing angle image 1102. Forexample, the electronic apparatus 1000 may determine the recommendedphotographing composition as a triangular composition based oninformation of a subject (e.g., the Taj Mahal) and surroundingenvironment information (e.g., midday and outdoors), and may determine arecommendation photographing area 1103 as an area where the Taj Mahal islocated at the center. In this case, according to an embodiment, thepreview image (e.g., information about the current photographing area1101) and the maximum viewing angle image 1102 may be input to therecommendation model 300, and an optimal photographing composition(e.g., information about the recommendation photographing area 1103) maybe output from the recommendation model 300.

The electronic apparatus 1000 may output information 1104 for guidingphotographing. For example, the electronic apparatus 1000 may output avoice message (e.g., “Move leftward, zoom out”) to obtain a photographcorresponding to the recommendation photographing area 1103, instead ofthe current photographing area 1101.

The electronic apparatus 1000 may graphically display the information1104 for guiding photographing, instead of the voice message. Anoperation by which the electronic apparatus 1000 graphically displaysthe information 1104 for guiding photographing will now be describedwith reference to FIGS. 12 and 15.

FIGS. 12 and 13 are views for describing an operation of displayinginformation for guiding a recommended photographing composition on amaximum viewing angle image according to various embodiments of thedisclosure.

Referring to 1200-1 of FIG. 12, the electronic apparatus 1000 maydisplay a preview image 1210 of an area (hereinafter, referred to as acurrent photographing area) currently photographed by a first camera.Also, the electronic apparatus 1000 may determine a recommendationphotographing area by analyzing a maximum viewing angle image obtainedby a second camera.

Referring to 1200-2 of FIG. 12, the electronic apparatus 1000 mayprovide information about the recommendation photographing area by usingthe maximum viewing angle image. For example, the electronic apparatus1000 may display a maximum viewing angle image 1202 on a screen. Theelectronic apparatus 1000 may display a first indicator 1201 indicatingthe current photographing area and a second indicator 1203 indicatingthe recommendation photographing area on the maximum viewing angle image1202. According to an embodiment, the preview image 1210 of the currentphotographing area may be displayed as a thumbnail image 1204.

Referring to 1300-1 of FIG. 13, a user may move the electronic apparatus1000 while watching the screen of the electronic apparatus 1000 so thatthe first indicator 1201 indicating the current photographing area mayapproach the second indicator 1203 indicating the recommendationphotographing area, and may adjust a zoom magnification. As theelectronic apparatus 1000 moves and the zoom magnification is adjusted,a position of the first indicator 1201 indicating the currentphotographing area on the maximum viewing angle image 1202 may bechanged. Also, as the current photographing area is changed, thethumbnail image 1204 may also be changed.

Referring to 1300-2 of FIG. 13, when the first indicator 1201 indicatingthe current photographing area is the same as the second indicator 1203indicating the recommendation photographing area, the electronicapparatus 1000 may no longer display the maximum viewing angle image1202, and may display a preview image 1301 of the current photographingarea that is the same as the recommendation photographing area on theentire screen. The user may check the preview image 1301 and may selecta photographing button. In this case, the electronic apparatus 1000 mayobtain a photograph of the recommendation photographing area.

According to another embodiment, when the first indicator 1201indicating the current photographing area is the same as the secondindicator 1203 indicating the recommendation photographing area, theelectronic apparatus 1000 may automatically perform photographing.

Although the electronic apparatus 1000 displays information for guidingthe recommended photographing composition on the maximum viewing angleimage 1202 in FIGS. 12 and 13, the disclosure is not limited thereto.The electronic apparatus 1000 may display the information for guidingthe recommended photographing composition in any of various ways. Anoperation by which the electronic apparatus 1000 displays theinformation for guiding the recommended photographing composition willnow be described with reference to FIGS. 14 and 15.

FIGS. 14 and 15 are views for describing an operation by which theelectronic apparatus 1000 displays information about a recommendedphotographing composition according to various embodiments of thedisclosure.

Referring to FIG. 14, the electronic apparatus 1000 may obtain a previewimage 1401 and surrounding environment information 1402 of surroundingsof a subject included in the preview image 1401. In this case, theelectronic apparatus 1000 may determine a recommended photographingcomposition by using the recommendation model 300. For example, when thesubject is a couple, a photographing place is outdoors, and aphotographing time is midday, the electronic apparatus 1000 maydetermine a composition where a center 1410 of the couple is located ata lower left intersection 1420 of a rule-of-thirds composition as therecommended photographing composition.

In this case, the electronic apparatus 1000 may display a graphicindicator 1403 for guiding the recommended photographing composition onthe preview image 1401. For example, the electronic apparatus 1000 maydisplay a rightwards arrow along with text telling a photographer ‘Moverightward’ on the preview image 1401.

Referring to FIG. 15, when the subject is a person, the electronicapparatus 1000 may provide a guide for causing the photographer to moveto the photographer, and may provide a guide for moving the subject. Thefollowing will be described on the assumption that the electronicapparatus 1000 determines a composition where a center 1520 of thecouple is located at a first intersection 1521 that is a lower leftintersection of a rule-of-thirds composition as the recommendedphotographing composition.

Referring to 1500-1 of FIG. 15, the electronic apparatus 1000 mayprovide a guide for causing the photographer to move rightward alongwith the electronic apparatus 1000 to the photographer by using cameraicons 1501 so that the center 1520 of the couple (e.g., shown in thepreview image 1510) is located at the first intersection 1521. Forexample, the electronic apparatus 1000 may display a white camera iconcorresponding to a current photographing composition, may display ablack camera icon corresponding to the recommended photographingcomposition, and then may display an arrow between the white camera iconand the black camera icon.

Referring to 1500-2 of FIG. 15, the electronic apparatus 1000 mayprovide a guide for moving the subject leftward by using person icons1502 so that the center 1520 of the couple is located at the firstintersection 1521. For example, the electronic apparatus 1000 maydisplay a white person icon corresponding to the current photographingcomposition, may display a black person icon corresponding to therecommended photographing composition, and then may display an arrowbetween the white person icon and the black person icon.

FIG. 16 is a flowchart for describing a method by which the electronicapparatus 1000 interoperates with a server 2000 to provide informationabout a recommended photographing composition according to an embodimentof the disclosure.

In operation S1610, the electronic apparatus 1000 may execute an AIassistant application. The AI assistant application may understood allinput methods such as a user's voice and touch, and a camera and mayprovide various services. For example, the AI assistant application(e.g., Bixby of Samsung) may recognize an object, an image, text, abarcode, etc. input to the camera and may provide information about arecommended photographing composition.

Referring to FIG. 17, the electronic apparatus 1000 may execute the AIassistant application in response to various user inputs. For example,referring to 1700-1 of FIG. 17, the electronic apparatus 1000 maydisplay an image 1701 for executing the AI assistant application on apreview image 1710. When the user selects (e.g., touches) the image1701, the electronic apparatus 1000 may execute the AI assistantapplication.

Referring to 1700-2 of FIG. 17, the electronic apparatus 1000 mayreceive the user's voice input (e.g., “Bixby! Take a photograph”) 1702.In this case, the electronic apparatus 1000 may execute the AI assistantapplication according to the user's voice input 1702.

Referring to 1700-3 of FIG. 17, the electronic apparatus 1000 mayreceive an input that selects a specific hardware button 1703 forrequesting to execute the AI assistant application. When the input thatselects the specific hardware button 1703 is received, the electronicapparatus 1000 may execute the AI assistant application.

Referring back to FIG. 16, in operation S1620, the electronic apparatus1000 may obtain information about a preview image and surroundingenvironment information of a subject through the AI assistantapplication.

According to an embodiment, the information about the preview image maybe the preview image itself, or may be feature information obtained fromthe preview image. According to an embodiment, the AI assistantapplication may extract the feature information from the preview imageby using a learning network model. For example, the feature informationobtained from the preview image may include, but is not limited to, acurrent photographing composition of the preview image, a type of thesubject, and a position of the subject.

According to an embodiment, the AI assistant application may obtain thesurrounding environment information of the subject by analyzing thepreview image. In this case, according to an embodiment, the AIassistant application may obtain the surrounding environment informationof the subject from the preview image by using the learning networkmodel.

According to an embodiment, the AI assistant application may obtain thesurrounding environment information of the subject by using at least onesensor. For example, the AI assistant application may obtain informationabout a current position of the subject and whether the subject islocated indoors or outdoors by using a position sensor. The AI assistantapplication may obtain an illuminance value of surroundings of thesubject by using an illuminance sensor. Also, the AI assistantapplication may determine whether the surroundings of the subject arecurrently in daytime or nighttime by using the illuminance sensor. TheAI assistant application may obtain temperature information of thesurroundings of the subject by using a temperature sensor, and mayobtain humidity information of the surroundings of the subject by usinga humidity sensor. The AI assistant application may obtain thesurrounding environment information by using a plurality of imagesensors (or a plurality of cameras).

In operation S1630, the electronic apparatus 1000 may transmit theinformation about the preview image and the surrounding environmentinformation of the subject to the server 2000. For example, the AIassistant application may transmit the information about the previewimage and the surrounding environment information of the subject to theserver 2000 through wired/wireless communication.

In operation S1640, the server 2000 may determine a recommendedphotographing composition based on the information about the previewimage and the surrounding environment information of the subject.According to an embodiment, the server 2000 may determine therecommended photographing composition by using the learning networkmodel (e.g., the recommendation model 300) trained based on photographstaken by professionals.

According to an embodiment, the server 2000 may determine therecommended photographing composition according to information of thesubject (e.g., a type of the identified subject and the number ofsubjects) and the surrounding environment information (e.g., whether aphotographing place is indoors or outdoors, a direction of light, anintensity of light, or a color temperature). For example, when thesubject is several people and there is a sunset outdoors as a resultobtained by analyzing the information of the subject and the surroundingenvironment information, the server 2000 may determine a compositionwhere the several people are clearly shown on a frame when there is asunset outdoors as the recommended photographing composition.

According to an embodiment, when the identified subject is a tallbuilding and a characteristic subject does not exist in the surroundingsas an analysis result based on a maximum viewing angle image, the server2000 may determine a vertical mode as the recommended photographingcomposition. Also, when the subject is several people and the peoplestand on a wide lawn outdoors as an analysis result based on the maximumviewing angle image, the server 2000 may determine a horizontal mode asthe recommended photographing composition.

According to an embodiment, the server 2000 may determine therecommended photographing composition in consideration of aphotographing area and a photographing angle. Accordingly, the server2000 may determine a recommendation photographing area and arecommendation photographing angle.

A method by which the server 2000 determines the recommendedphotographing composition based on the information of the subject andthe surrounding environment information may correspond to a method bywhich the electronic apparatus 1000 determines the recommendedphotographing composition based on the information of the subject andthe surrounding environment information, and thus a detailed explanationthereof will not be given.

In operation S1650, the server 2000 may transmit information about therecommended photographing composition to the electronic apparatus 1000.For example, the server 2000 may transmit the recommendationphotographing area, the recommendation photographing angle, and therecommended photographing composition to the electronic apparatus 1000through wired/wireless communication. According to an embodiment, theelectronic apparatus 1000 may receive the information about therecommended photographing composition from the server 2000 through theAI assistant application.

In operation S1660, the electronic apparatus 1000 may provide theinformation about the recommended photographing composition through theAI assistant application.

According to an embodiment, the electronic apparatus 1000 may determinethe current photographing composition by using the preview image. Inthis case, when a similarity between the current photographingcomposition and the recommended photographing composition is less than athreshold value, the electronic apparatus 1000 may provide theinformation about the recommended photographing composition.

According to an embodiment, the electronic apparatus 1000 may providethe information about the recommended photographing composition byoutputting information for guiding the recommended photographingcomposition. In this case, the electronic apparatus 1000 may output theinformation for guiding the recommended photographing composition byusing at least one of, but not limited to, a video signal, an audiosignal, and a vibration signal. For example, the electronic apparatus1000 may output a voice for guiding the recommended photographingcomposition. Also, the electronic apparatus 1000 may display a graphicindicator for guiding the recommended photographing composition on animage having a viewing angle (e.g., the maximum viewing angle image)greater than that of the preview image.

Although the server 2000 determines the recommended photographingcomposition based on the information of the subject and the surroundingenvironment information of the subject in FIG. 16, the disclosure is notlimited thereto. According to an embodiment, the server 2000 maydetermine a recommendation photographing setting value based on theinformation of the subject and the surrounding environment informationof the subject and may transmit information about the recommendationphotographing setting value to the electronic apparatus 1000.

Some operations of FIG. 16 may be omitted. For example, the electronicapparatus 1000 may transmit only the information about the previewimage, without transmitting the surrounding environment information ofthe subject, to the server 2000. In this case, the server 2000 maydirectly obtain the surrounding environment information of the subjectbased on the information about the preview image.

FIG. 18 is a flowchart for describing a method of providing informationabout a plurality of recommended photographing compositions according toan embodiment of the disclosure.

In operation S1810, the electronic apparatus 1000 may identify a subjectincluded in a preview image. For example, the electronic apparatus 1000may identify the subject included in the preview image by analyzing thepreview image. In this case, according to an embodiment, the electronicapparatus 1000 may identify the subject included in the preview image byusing a learning network model (e.g., an AI model) of an AI system.Operation S1810 corresponds to operation S210 of FIG. 2, and thus adetailed explanation thereof will not be given.

In operation S1820, the electronic apparatus 1000 may obtain surroundingenvironment information of the subject. For example, the electronicapparatus 1000 may obtain information related to light in surroundingsof the subject as the surrounding environment information.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information of the subject by analyzing thepreview image. In this case, according to an embodiment, the electronicapparatus 1000 may obtain the surrounding environment information of thesubject from the preview image by using the learning network model ofthe AI system.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information of the subject by using at least onesensor. For example, the electronic apparatus 1000 may obtaininformation about a current position of the subject and whether thesubject is located indoors or outdoors by using a position sensor. Theelectronic apparatus 1000 may obtain an illuminance value of thesurroundings of the subject by using an illuminance sensor. Also, theelectronic apparatus 1000 may determine whether the surroundings of thesubject are currently in daytime or nighttime by using the illuminancesensor. The electronic apparatus 1000 may obtain temperature informationof the surroundings of the subject by using a temperature sensor, andmay obtain humidity information of the surroundings of the subject byusing a humidity sensor. The electronic apparatus 1000 may obtain thesurrounding environment information by using a plurality of imagesensors (or a plurality of cameras). Operation S1820 corresponds tooperation S220 of FIG. 2, and thus a detailed explanation thereof willnot be given.

In operation S1830, the electronic apparatus 1000 may determine aplurality of recommended photographing compositions by using informationof the subject and the surrounding environment information of thesubject. According to an embodiment, the electronic apparatus 1000 maydetermine the plurality of recommended photographing compositions byusing the learning network model (e.g., the recommendation model 300)trained based on photographs taken by professionals.

According to an embodiment, the server 2000 may determine the pluralityof recommended photographing compositions according to the informationof the subject (e.g., a type of the identified subject and the number ofsubjects) and the surrounding environment information (e.g., whether aphotographing place is outdoors or indoors, a direction of light, anintensity of light, and a color temperature).

In operation S1840, the electronic apparatus 1000 may provideinformation about the plurality of recommended photographingcompositions.

According to an embodiment, the electronic apparatus 1000 may providethe information about the plurality of recommended photographingcompositions by outputting information for guiding the plurality ofrecommended photographing compositions. In this case, the electronicapparatus 1000 may output the information for guiding the plurality ofrecommended photographing compositions by using at least one of, but notlimited to, a video signal, an audio signal, and a vibration signal. Forexample, the electronic apparatus 1000 may output a voice for guidingthe plurality of recommended photographing compositions. Also, theelectronic apparatus 1000 may display a plurality of graphic indicatorsfor guiding the plurality of recommended photographing compositions onthe preview image. The electronic apparatus 1000 may display theplurality of graphic indicators (e.g., box images) for guiding theplurality of recommended photographing compositions on an image having aviewing angle (e.g., a maximum viewing angle image) greater than that ofthe preview image.

According to an embodiment, the electronic apparatus 1000 may displaythumbnail images respectively corresponding to the plurality ofrecommended photographing compositions on one screen. According toanother embodiment, the electronic apparatus 1000 may display previewimages respectively corresponding to the plurality of recommendedphotographing compositions on different pages.

In operation S1850, the electronic apparatus 1000 may receive a userinput that selects one of the plurality of recommended photographingcompositions. According to an embodiment, the electronic apparatus 1000may obtain a photograph image corresponding to the recommendedphotographing composition selected by a user.

For example, the electronic apparatus 1000 may receive an input thatmatches a first indicator corresponding to a current photographingcomposition to a second indicator corresponding to a second recommendedphotographing composition from among the plurality of recommendedphotographing compositions. In this case, the electronic apparatus 1000may start photographing and may obtain a photograph image correspondingto the second recommended photographing composition.

According to an embodiment, the electronic apparatus 1000 may obtain aphotograph image corresponding to the recommended photographingcomposition selected by the user. An operation by which the electronicapparatus 1000 provides the information about the plurality ofrecommended photographing compositions will now be described withreference to FIGS. 19 through 21.

FIGS. 19, 20, and 21 are views for describing an operation by which theelectronic apparatus 1000 displays information about a plurality ofrecommended photographing compositions according to various embodimentsof the disclosure.

Referring to FIG. 19, the electronic apparatus 1000 may determine aplurality of recommended photographing compositions, and may displaygraphic indicators for guiding the plurality of recommendedphotographing compositions on a preview image.

According to an embodiment, the electronic apparatus 1000 may displaythe preview image obtained by a first camera on a screen. In this case,the preview image may include a couple as a subject and may include afield and a mountain as a background.

The electronic apparatus 1000 may determine a rule-of-thirds compositionas a recommended photographing composition. In particular, theelectronic apparatus 1000 may determine a composition where a center1911 of the subject is located at a first point 1901 as a firstrecommended photographing composition and a composition where a center1911 of the subject is located at a second point 1902 as a secondrecommended photographing composition, from among virtual points wherehorizontal/vertical lines meet, in consideration of information of thesubject and surrounding environment information.

The electronic apparatus 1000 may display a first indicator 1910 forguiding the first recommended photographing composition and a secondindicator 1920 for guiding the second recommended photographingcomposition on the preview image. For example, the first indicator 1910may include an icon (e.g., a leftwards arrow) for guiding a photographerto move leftward. The second indicator 1920 may include an icon (e.g., arightwards arrow) for guiding the photographer to move rightward.

Referring to FIG. 20, the electronic apparatus 1000 may displaythumbnail images respectively corresponding to a plurality ofphotographing compositions on a screen.

Referring to 2000-1 of FIG. 20, the electronic apparatus 1000 maydisplay an icon 2001 for requesting for a recommended photographingcomposition on a preview image 2010 obtained by a first camera. Theelectronic apparatus 1000 may receive a user input that selects the icon2001 for requesting for the recommended photographing composition. Forexample, when a user (photographer) wants to obtain information aboutthe recommended photographing composition, the user may touch the icon2001 displayed on the preview image 2010.

Referring to 2000-2 of FIG. 20, the electronic apparatus 1000 maydetermine a plurality of recommended photographing compositions inresponse to an input that touches the icon 2001. In this case, theelectronic apparatus 1000 may display thumbnail images corresponding tothe plurality of recommended photographing compositions.

For example, the electronic apparatus 1000 may determine arule-of-thirds composition as the recommended photographing composition.In particular, the electronic apparatus 1000 may determine a compositionwhere a center of a subject is located at the first point 1901 as afirst recommended photographing composition, may determine a compositionwhere the center of the subject is located at a second point as a secondrecommended photographing composition, and may determine a compositionwhere the center of the subject is located at a third point as a thirdrecommended photographing composition, from among virtual points wherehorizontal/vertical lines meet, in consideration of information of thesubject and surrounding environment information.

In this case, the electronic apparatus 1000 may obtain thumbnail imagescorresponding to the recommended photographing compositions from amaximum viewing angle image obtained by a second camera. For example,the electronic apparatus 1000 may obtain a first thumbnail image 2003corresponding to the first recommended photographing composition, asecond thumbnail image 2004 corresponding to the second recommendedphotographing composition, and a third thumbnail image 2005corresponding to the third recommended photographing composition fromthe maximum viewing angle image.

The electronic apparatus 1000 may display the first thumbnail image 2003corresponding to the first recommended photographing composition, thesecond thumbnail image 2004 corresponding to the second recommendedphotographing composition, and the third thumbnail image 2005corresponding to the third recommended photographing composition on thescreen along with a thumbnail image 2002 corresponding to the previewimage 2010. In this case, since the user (photographer) may checkphotograph images according to the plurality of recommendedphotographing compositions and a photograph image according to a currentphotographing composition in advance, the user may easily select adesired photographing composition.

Referring to FIG. 21, when a plurality of recommended photographingcompositions are determined, the electronic apparatus 1000 may obtainpreview images respectively corresponding to recommended photographingcompositions on a maximum viewing angle image obtained by a secondcamera. The electronic apparatus 1000 may display the preview imagesrespectively corresponding to the plurality of recommended photographingcompositions on different pages.

Referring to 2100-1 of FIG. 21, the electronic apparatus 1000 maydisplay a first preview image corresponding to a first recommendedphotographing composition on a first page 2110. In this case, theelectronic apparatus 1000 may display a first indicator 2101 for guidingthe first recommended photographing composition. For example, the firstindicator 2101 may include an arrow for guiding a photographer to moverightward.

Referring to 2100-2 of FIG. 21, the electronic apparatus 1000 maydisplay a second preview image corresponding to a second recommendedphotographing composition on a second page 2120. In this case, theelectronic apparatus 1000 may display a second indicator 2102 forguiding the second recommended photographing composition. For example,the second indicator 2102 may include an arrow for guiding thephotographer to move leftward.

Referring to 2100-3 of FIG. 21, the electronic apparatus 1000 maydisplay a third preview image corresponding to a third recommendedphotographing composition on a third page 2130. In this case, theelectronic apparatus 1000 may display a third indicator 2103 for guidingthe third recommended photographing composition. For example, the thirdindicator 2103 may include an image for guiding the photographer to movedownward the electronic apparatus 1000 and zoom out.

The user (photographer) may check preview images according to aplurality of recommended photographing compositions in advance, and mayeasily determine a desired photographing composition. Also, the user(photographer) may easily obtain a photograph of a desired photographingcomposition according to an indicator for guiding a recommendedphotographing composition.

A method by which the electronic apparatus 1000 recommends a pose when asubject is a person will now be described in detail with reference toFIGS. 22 through 26.

FIG. 22 is a flowchart for describing a method of providing informationabout a recommended pose according to an embodiment of the disclosure.

In operation S2210, the electronic apparatus 1000 may identify a subjectincluded in a preview image. For example, the electronic apparatus 1000may identify the subject included in the preview image by analyzing thepreview image. In this case, according to an embodiment, the electronicapparatus 1000 may identify the subject included in the preview image byusing a learning network model of an AI system. Operation S2210corresponds to operation S210 of FIG. 2, and thus a detailed explanationthereof will not be given.

In operation S2220, the electronic apparatus 1000 may obtain surroundingenvironment information of the subject.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information of the subject by analyzing thepreview image. In this case, according to an embodiment, the electronicapparatus 1000 may obtain the surrounding environment information of thesubject from the preview image by using the learning network model ofthe AI system.

According to an embodiment, the electronic apparatus 1000 may obtain thesurrounding environment information of the subject by using at least onesensor. For example, the electronic apparatus 1000 may obtaininformation about a current position of the subject and whether thesubject is located indoors or outdoors by using a position sensor. Theelectronic apparatus 1000 may obtain an illuminance value ofsurroundings of the subject by using an illuminance sensor. Also, theelectronic apparatus 1000 may determine whether the surroundings of thesubject are currently in daytime or nighttime by using the illuminancesensor. The electronic apparatus 1000 may obtain temperature informationof the surroundings of the subject by using a temperature sensor, andmay obtain humidity information of the surroundings of the subject byusing a humidity sensor. The electronic apparatus 1000 may obtain thesurrounding environment information by using a plurality of imagesensors (or a plurality of cameras). Operation S2220 corresponds tooperation S220 of FIG. 2, and thus a detailed explanation thereof willnot be given.

In operation S2230, the electronic apparatus 1000 may determine arecommended pose by using information of the subject and the surroundingenvironment information of the subject.

According to an embodiment, the electronic apparatus 1000 may determinethe recommended pose by using the learning network model. The learningnetwork model may be generated by using data obtained by learning a pose(recognition) of a person, a background, and presence of another personbased on photographs of the person for learning. The recommended posemay include not only a whole-body pose of the person but also a facialexpression or a face angle.

According to an embodiment, when the person's face is recognized througha first camera, the electronic apparatus 1000 may select a pose cardsuitable for a current surrounding environment of the subject form amonga plurality of pose cards that are pre-generated. The electronicapparatus 1000 may determine a pose included in the selected pose cardas the recommended pose.

According to an embodiment, the electronic apparatus 1000 may determinethe recommended pose by further considering personalized learning dataof a user. For example, when the electronic apparatus 1000 recommends asitting pose, a standing pose, and a lying pose to a first user, but theuser takes the sitting pose in all photographs, the electronic apparatus1000 may determine the recommended pose based on the sitting pose. Also,when all of photographs where the user takes the lying pose are deleted,the electronic apparatus 1000 may not select the lying pose whendetermining the recommended pose.

According to an embodiment, the electronic apparatus 1000 may determinea plurality of recommended poses by using the information of the subjectand the surrounding environment information of the subject. For example,when a plurality of subjects stand on a lawn, the electronic apparatus1000 may recognize a plurality of people as subject information and mayrecognize outdoors, a lawn, and midday as surrounding environmentinformation. In this case, the electronic apparatus 1000 may select afirst pose card, a second pose card, and a third pose card related tophotographs of the plurality of people standing on the lawn in middayfrom among a plurality of pose cards that are pre-generated.

In operation S2240, the electronic apparatus 1000 may compare a currentpose with the recommended pose and may determine whether a similaritybetween the current pose and the recommended pose is less than athreshold value.

According to an embodiment, the electronic apparatus 1000 may determinea current pose of a person who is the subject by analyzing the previewimage. According to an embodiment, the electronic apparatus 1000 maydetermine the current pose of the person who is the subject by using thelearning network model of the AI system.

Examples of the pose may include, but are not limited to, a pose oflying to the left, a pose of lying to the right, a pose of sitting downwith legs stretched out, a pose of sitting down with knees bent andsitting down, a pose of bending the waist, a pose of raising hands onthe waist, a pose of standing while looking forward, a pose of standingwhile looking at the side, a pose of standing with his/her back, and apose of jumping.

According to an embodiment, examples of the pose may further include,but are not limited to, a shape of a finger (e.g., a V-shape, a heartshape, a fisted shape, or a thumbs up shape), a facial expression (e.g.,no expression, a smiling expression, a crying expression, or a frowningexpression), and whether accessories are worn (e.g., whether glasses areworn, whether sunglasses are worn, or whether a hat is worn).

According to an embodiment, the electronic apparatus 1000 may comparethe current pose with the recommended pose. If a similarity between thecurrent pose and the recommended pose is equal to or greater than athreshold value (e.g., 97%), the electronic apparatus 1000 may notprovide information about the recommended pose.

In operation S2250, the electronic apparatus 1000 may provide theinformation about the recommended pose when the similarity between thecurrent pose and the recommended pose is less than the threshold value.

According to an embodiment, the electronic apparatus 1000 may outputinformation for guiding the recommended pose. For example, theelectronic apparatus 1000 may output the information for guiding therecommended pose by using at least one of, but not limited to, a videosignal, an audio signal, and a vibration signal.

According to an embodiment, the electronic apparatus 1000 may display aline for guiding the recommended pose on the subject. According to anembodiment, the electronic apparatus 1000 may display a graphic indictor(e.g., an icon) for guiding the recommended pose on a portion of thepreview image (e.g., an upper right portion).

According to an embodiment, the electronic apparatus 1000 may provide aplurality of recommended pose cards to a predetermined portion when aplurality of recommended poses are determined.

An operation by which the electronic apparatus 1000 provides theinformation about the recommended pose will now be described withreference to FIGS. 23 through 26.

FIG. 23 is a view for describing an operation by which the electronicapparatus 1000 displays information about a recommended pose accordingto an embodiment of the disclosure.

Referring to FIG. 23, the electronic apparatus 1000 may analyze apreview image obtained by a first camera and may recognize that a mainsubject 2301 is one person and the main subject 2301 is awarded in afilm festival (indoors) that is a surrounding environment. Also, theelectronic apparatus 1000 may recognize that the main subject 2301stands with a stiff look (a current pose). In this case, the electronicapparatus 1000 may determine a pose suitable for an awards ceremony hall(indoors) as a recommended pose. The electronic apparatus 1000 maydisplay information about the recommended pose on a screen.

For example, referring to 2300-1 of FIG. 23, the electronic apparatus1000 may determine a pose of raising hands on the waist as therecommended pose. The electronic apparatus 1000 may display lines 2302for guiding the recommended pose so that the lines 2302 overlap an imageof the main subject 2301. In this case, a photographer may check therecommended pose and may request the main subject 2301 to change a pose.

Referring to 2300-2 of FIG. 23, the electronic apparatus 1000 maydisplay an icon corresponding to the recommended pose on a specificportion 2303 so that the icon does not overlap the image of the mainsubject 2301. In this case, the photographer may check the recommendedpose displayed on the specific portion 2303.

FIG. 24 is a view for describing an operation of providing informationabout a recommended pose by using the number of subjects and surroundingenvironment information of the subjects according to an embodiment ofthe disclosure.

Referring to 2400-1 of FIG. 24, the electronic apparatus 1000 may obtaininformation indicating that the number of subjects is 3 by analyzing apreview image 2410. Also, the electronic apparatus 1000 may find thatthe three subjects currently exist in daytime outdoors by analyzing thepreview image or by using a sensor. In this case, the electronicapparatus 1000 may determine a pose that may be taken by the threesubjects in daytime outdoors as a recommended pose, by using therecommendation model 300. For example, the electronic apparatus 1000 maydetermine a pose where the three subjects raise their hands and givecheers as the recommended pose. The electronic apparatus 1000 mayprovide a graphic indicator for guiding the recommended pose. Forexample, the electronic apparatus 1000 may display an icon image 2401corresponding to the recommended pose on the preview image.

Referring to 2400-2 of FIG. 24, the electronic apparatus 1000 may obtaininformation indicating that the number of subjects is 3 by analyzing apreview image 2420. Also, the electronic apparatus 1000 may find thatthe three subjects exist indoors by analyzing the preview image or byusing a sensor. In this case, the electronic apparatus 1000 maydetermine a pose that may be taken by the three subjects standingindoors as a recommended pose. For example, the electronic apparatus1000 may determine a pose where a middle subject among the threesubjects sits down and two other subjects on both sides stand up as therecommended pose. The electronic apparatus 1000 may provide a graphicindicator for guiding the recommended pose. For example, the electronicapparatus 1000 may display an icon image 2402 corresponding to therecommended pose on the preview image.

FIG. 25 is a view for describing an operation of providing informationabout a plurality of recommended poses according to an embodiment of thedisclosure.

Referring to FIG. 25, the electronic apparatus 1000 may obtaininformation indicating that the number of subjects is 1 and a place is agraduation ceremony hall by analyzing a preview image. In this case, theelectronic apparatus 1000 may determine a pose that may be taken by thesubject in the graduation ceremony hall as a recommended pose. Forexample, the electronic apparatus 1000 may determine various poses suchas a pose of jumping and throwing a graduation cap, a pose of puttingone hand on the waist and raising the other hand, or a pose of leaningagainst the wall as the recommended poses.

According to an embodiment, the electronic apparatus 1000 may displayinformation (e.g., 2500) for guiding the plurality of recommended poseson a specific portion 2501 of the displayed information 2500. Forexample, the electronic apparatus 1000 may display icon imagesrespectively corresponding to the plurality of recommended poses on thespecific portion 2501.

When there are the plurality of recommended poses, the electronicapparatus 1000 may not display all icons indicating the plurality ofrecommended poses on the specific portion 2501. In this case, theelectronic apparatus 1000 may indicate that there are more recommendedposes other than those displayed on the specific portion 2501, byproviding an arrow to the specific portion 2501. A user may check theother recommended poses not shown on the specific portion 2501 bytouching the arrow or dragging his/her finger to the left or rightwithin the specific portion 2501. For example, when a drag input isreceived in a state where a first icon (e.g., an icon corresponding to apose of jumping) corresponding to a first recommended pose is displayedon the specific portion 2501, the electronic apparatus 1000 may displaya second icon (e.g., a pose of standing with both hands on the waist)corresponding to a second recommended pose, instead of the first icon,on the specific portion 2501. A method by which the electronic apparatus1000 provides information about the plurality of recommended poses isnot limited to that of FIG. 25 and any of various other methods may beused.

FIG. 26 is a view for describing an operation of recommending an optimalface composition according to an embodiment of the disclosure. FIG. 26will be described on the assumption that a user of the electronicapparatus 1000 takes a selfie image.

Referring to FIG. 26, the electronic apparatus 1000 may obtain a previewimage including a user's face image. The electronic apparatus 1000 mayobtain surrounding environment information (e.g., a direction oflighting or the impression of color of lighting) by analyzing thepreview image or by using sensors. In this case, the electronicapparatus 1000 may input information about the user's face image and thesurrounding environment information to the recommendation model 300. Inthis case, the electronic apparatus 1000 may obtain information about anoptimal face composition 2600 from the recommendation model 300.

The recommendation model 300 may be a model trained by usingpersonalized learning data. For example, the recommendation model 300may be a model trained based on photographs usually uploaded to a socialnetworking service (SNS) server and photographs deleted by the user froma memory after photographing. The recommendation model 300 may haveinformation indicating that the user deletes photographs where the leftside of the face is highlighted and uploads photograph where the rightside of the face is highlighted to the SNS server.

Accordingly, when a preview image 2601 and surrounding environmentinformation 2602 are input to the recommendation model 300, therecommendation model 300 may determine a recommendation face angle byusing the personalized learning data. For example, the recommendationmodel 300 may determine a face composition where the right face of theface is highlighted and a gaze is downward as the recommendation facecomposition.

Since a current composition of the face included in the preview image isa composition where the left side is highlighted, the electronicapparatus 1000 may provide information about the recommendation facecomposition. For example, the electronic apparatus 1000 may display anicon image 2603 corresponding to the recommendation face composition ona screen. Also, the electronic apparatus 1000 may output a voice signalcorresponding to the recommendation face composition. For example, theelectronic apparatus 1000 may output a voice message saying that “You'dbetter turn your face in the opposite direction, tilt your chin down,and shift your gaze downward”.

FIGS. 27 and 28 are block diagrams for describing the electronicapparatus 1000 according to various embodiments of the disclosure.

Referring to FIG. 27, the electronic apparatus 1000 according to anembodiment may include an output interface 1100, a sensor 1200, and aprocessor 1300. However, all elements illustrated in FIG. 27 are notessential elements. The electronic apparatus 1000 may include more orfewer than the elements illustrated in FIG. 27.

For example, as shown in FIG. 28, the electronic apparatus 1000according to an embodiment may include a communication interface 1400,an A/V input interface 1500, a user input interface 1600, and a memory1700 in addition to the output interface 1100, the sensor 1200, and theprocessor 1300.

The elements will be sequentially described below.

The output interface 1100 for outputting an audio signal, a videosignal, or a vibration signal may include a display 1111, a sound outputinterface 1112, and a vibration motor 1113.

The sound output interface 1112 outputs audio data received from thecommunication interface 1400 or stored in the memory 1700. Also, thesound output interface 1112 outputs a sound signal related to a function(e.g., a call signal receiving sound, a message receiving sound, or anotification sound) performed by the electronic apparatus 1000. Thesound output interface 1112 may include a speaker or a buzzer.

The vibration motor 1113 may output a vibration signal. For example, thevibration motor 1113 may output a vibration signal corresponding to anoutput of audio data or video data (e.g., a call signal receiving soundor a message receiving sound). Also, the vibration motor 1113 may outputa vibration signal when a touch is input to a touchscreen. The vibrationmotor 1113 may output a vibration signal when a current position is thesame as a recommendation composition when a photographer moves or asubject moves.

The output interface 1100 may display a preview image including thesubject recognized by a first camera 1511. The output interface 1100 mayprovide information related to a recommended photographing composition.For example, the output interface 1100 may display a graphic indicatorfor guiding the recommended photographing composition.

The output interface 1100 may provide information related to a pluralityof recommended photographing compositions. For example, the outputinterface 1100 may display thumbnail images respectively correspondingto the plurality of recommended photographing compositions and mayprovide preview images respectively corresponding to the plurality ofrecommended photographing compositions on different pages.

The output interface 1100 may provide information at least one of arecommendation photographing area, a recommendation photographing angle,a recommended pose, and a recommendation face composition.

The sensor 1200 may include at least one from among, but not limited to,a terrestrial magnetism sensor 1211, an acceleration sensor 1212, a tiltsensor 1213, an infrared sensor 1214, a gyroscope sensor 1215, aposition sensor (e.g., a GPS) 1216, a temperature/humidity sensor 1217,a proximity sensor 1218, and a light sensor 1219. Functions of thesensors may be intuitively derived by one of ordinary skill in the artfrom their names, and thus a detailed explanation thereof will not begiven.

The sensor 1200 may obtain surrounding environment information (e.g.,information related to light in surroundings of the subject) of thesubject. For example, the light sensor 1219 (or an illuminance sensor)may obtain an illuminance value of the surroundings of the subject.Also, the light sensor 1219 (or the illuminance sensor) may determinewhether the surroundings of the subject are currently in daytime ornighttime. The temperature/humidity sensor 1217 may obtain temperatureinformation of the surroundings of the subject and humidity informationof the surroundings of the subject.

The processor 1300 generally controls an overall operation of theelectronic apparatus 1000. For example, the processor 1300 may controloperations of the output interface 1100, the sensor 1200, thecommunication interface 1400, the A/V input interface 1500, the userinput interface 1600, and the memory 1700 by executing programs storedin the memory 1700.

According to an embodiment, the processor 1300 may include, but is notlimited to, an AI processor for generating a learning network model.According to an embodiment, the AI processor may be implemented as achip separate from the processor 1300.

The processor 1300 may identify the subject included in the previewimage and may determine a recommended photographing composition by usinginformation of the identified subject and the surrounding environmentinformation of the subject. The processor 1300 may provide informationrelated to the recommended photographing composition through the outputinterface 1100.

According to an embodiment, the processor 1300 may determine a currentphotographing composition from the preview image. When a similaritybetween the recommended photographing composition and the currentphotographing composition is less than a threshold value, the processor1300 may provide the information about the recommended photographingcomposition. When the similarity between the recommended photographingcomposition and the current photographing composition is less than thethreshold value, the processor 1300 may not provide the informationabout the recommended photographing composition.

The processor 1300 may obtain a first image having a viewing anglegreater than that of the preview image by using a second camera 1512different from the first camera 1511. The processor 1300 may obtain thesurrounding environment information of the subject by analyzing theobtained first image. The processor 1300 may control the outputinterface 1100 to display information for guiding the recommendedphotographing composition on the first image.

The processor 1300 may determine a plurality of recommendedphotographing compositions by using the information of the identifiedsubject and the surrounding environment information of the subject. Theprocessor 1300 may control the output interface 1100 to displaythumbnail images respectively corresponding to the plurality ofrecommended photographing compositions.

The processor 1300 may recommend or automatically apply a photographingsetting value based on the information of the identified subject and thesurrounding environment information of the subject. The processor 1300may determine a recommended pose of the subject by using the informationof the identified subject and the surrounding environment information ofthe subject. The processor 1300 may provide information about therecommended pose.

The communication interface 1400 may include one or more elements thatenable communication between the electronic apparatus 1000 and awearable device or between the electronic apparatus 1000 and the server2000. For example, the communication interface 1400 may include ashort-range communication interface 1411, a mobile communicationinterface 1412, and a broadcast receiver 1413.

Examples of the short-range communication interface 1411 may include,but are not limited to, a Bluetooth communication interface, a Bluetoothlow energy (BLE) communication interface, a near-field communicationinterface, a WLAN (Wi-Fi) communication interface, a Zigbeecommunication interface, an infrared data association (IrDA)communication interface, a Wi-Fi Direct (WFD) communication interface,an ultra-wideband (UWB) communication interface, and an Ant+communication interface.

The mobile communication interface 1412 transmits/receives a wirelesssignal to/from at least one of a base station, an external terminal, andthe server 2000 via a mobile communication network. Examples of thewireless signal may include a voice call signal, a video call signal,and any of various pieces of data according to text/multimedia messagetransmission/reception.

The broadcast receiver 1413 receives a broadcast signal and/orbroadcast-related information from the outside through a broadcastchannel. Examples of the broadcast channel may include a satellitechannel and a terrestrial channel. According to an embodiment, theelectronic apparatus 1000 may not include the broadcast receiver 1413.

According to an embodiment, the communication interface 1400 maytransmit information about the preview image and the surroundingenvironment information of the subject to the server 2000 connected tothe electronic apparatus 1000. The communication interface 1400 mayreceive, from the server 2000, information about a recommendedphotographing composition determined by using the information about thepreview image and the surrounding environment information of thesubject. The communication interface 1400 may upload a photograph imageto an SNS server or may download a photograph image from the SNS server.

The A/V input interface 1500 for receiving an audio signal input or avideo signal input may include the first camera 1511, the second camera1512, and a microphone 1513. The first camera 1511 and the second camera1512 may obtain image frames such as a still image or a moving image byusing an image sensor in a video call mode or a photographing mode. Animage captured by the image sensor may be processed by the processor1300 or an additional image processor (not shown).

The image frames processed by the first camera 1511 or the second camera1512 may be stored in the memory 1700 or may be transmitted to theoutside through the communication interface 1400.

According to an embodiment, the first camera 1511 may be a generalcamera, and the second camera 1512 may be at least one of, but notlimited to, a telephoto camera, a wide-angle camera, and a generalcamera. The first camera 1511 may correspond to the first camera 1001 ofFIG. 10, and the second camera 1512 may correspond to the second camera1002 of FIG. 10.

The microphone 1513 receives an external sound signal and processes theexternal sound signal into electrical voice data. For example, themicrophone 1513 may receive a sound signal from an external device or auser. The microphone 1513 may use any of various noise removingalgorithms to remove noise occurring when receiving the external soundsignal.

The user input interface 1600 is a unit through which the user inputsdata for controlling the electronic apparatus 1000. Examples of the userinput interface 1600 may include, but are not limited to, a keypad, adome switch, a touchpad (e.g., a contact-type capacitance method, apressure-type resistance film method, an infrared sensing method, asurface ultrasound transmission method, an integral tension measuringmethod, or a piezoelectric effect method), a jog wheel, and a jugswitch.

The memory 1700 may store a program for processing and controlling theprocessor 1300, and may store input/output data (e.g., the previewimage, a photograph image 1712, metadata, the surrounding environmentinformation, personalized learning data, and a recommended pose card).

The memory 1700 may include at least one type of storage medium fromamong a flash memory type memory, a hard disk type memory, a multimediacard micro type memory, a card-type memory (e.g., a secure digital (SD)memory or an extreme digital (XD) memory), a random-access memory (RAM),a static random-access memory (SRAM), a read-only memory (ROM), anelectrically erasable programmable read-only memory (EEPROM), aprogrammable read-only memory (PROM), a magnetic memory, a magneticdisk, and an optical disk.

Programs stored in the memory 1700 may be classified into a plurality ofmodules according to functions of the memory 1700, for example, arecommendation model 1711. The recommendation model 1711 corresponds tothe recommendation model 300 of FIG. 3, and thus a detailed explanationthereof will not be given. A process of generating the recommendationmodel 1711 will now be described with reference to FIGS. 29 through 32.

FIG. 29 is a block diagram of the processor 1300 according to anembodiment of the disclosure.

Referring to FIG. 29, the processor 1300 according to some embodimentsmay include a data learner 1310 and a data recognizer 1320.

The data learner 1310 may learn a standard for determining arecommendation situation (e.g., recommendation of a photographingcomposition, recommendation of a photographing area, recommendation of apose, or recommendation of an optimal face angle). The data learner 1310may learn a standard about which data is to be used in order todetermine a recommendation situation and how to determine the situationby using the data. The data learner 1310 may learn the standard fordetermining a recommendation situation by obtaining data (e.g., animage) to be used for learning (or training) and applying the obtaineddata to a data recognition model.

According to an embodiment, the data learner 1310 may learn acomposition where a subject (e.g., a person, an animal, a naturalobject, or a natural phenomenon) is photographed in a specificenvironment (e.g., indoors or outdoors, a general illuminance or a lowilluminance, daytime or nighttime, a sunset or a sunrise, winter orsummer, surroundings of a specific building, or surroundings of aspecific natural object). According to an embodiment, the data learner1310 may learn a photographing composition according to information ofthe subject (e.g., a type of the subject, the number of subjects, agender, and an age) or surrounding environment information of thesubject (e.g., the amount of light, a direction of light, an intensityof light, whether a location is indoors or outdoors, and whether it issurroundings of a specific building). For example, the data learner 1310may learn a composition of photographing a person, a composition ofphotographing an animal, a composition of photographing a mountain, acomposition of photographing a sea, a composition of photographing aperson indoors, a composition of photographing a person outdoors, acomposition of photographing a specific building outdoors, a compositionof photographing a food indoors, a composition of photographing severalpeople, a composition of photographing one person, a composition ofphotographing in nighttime, and a composition of photographing indaytime.

According to an embodiment, the data learner 1310 may learn personalizeddata. For example, the data learner 1310 may learn data aboutphotographing compositions of deleted photographs, data aboutphotographing compositions of photographs uploaded to an SNS server,data about photographing compositions of photographs transmitted todevices of friends, and data about a photographing composition of aphotograph designated as a profile image.

According to an embodiment, the data learner 1310 may learn a standardfor recommending a photographing area. According to an embodiment, whenthe subject is a person, the data learner 1310 may learn a standard forrecommending a pose. The data learner 1310 may learn a face composition.

According to an embodiment, the data learner 1310 may learn a standardfor recommending a photographing setting value. For example, the datalearner 1310 may learn photographing setting values obtained fromphotographs taken by professionals.

The data recognizer 1320 may determine a recommendation situation basedon data. The data recognizer 1320 may recognize the recommendationsituation from detected data by using the trained data recognitionmodel. The data recognizer 1320 may obtain image data (e.g., a previewimage) according to a standard preset by learning and may determine therecommendation situation based on the image data by using the datarecognition model by using the obtained image data as an input value.Also, a resultant value output by the data recognition model by usingthe obtained image data as an input value may be used to refine the datarecognition model.

At least one of the data learner 1310 and the data recognizer 1320 maybe manufactured as at least one hardware chip and may be mounted on theelectronic apparatus 1000. For example, at least one of the data learner1310 and the data recognizer 1320 may be manufactured as a dedicatedhardware chip for AI, or may be manufactured as a part of an existinggeneral-purpose processor (e.g., a central processing unit (CPU) or anapplication processor) or a graphics processor (e.g., a graphicsprocessing unit (GPU)) and may be mounted on the electronic apparatus1000.

In this case, the data learner 1310 and the data recognizer 1320 may bemounted on one electronic apparatus 1000, or may be separately mountedon electronic apparatuses. For example, one of the data learner 1310 andthe data recognizer 1320 may be included in the electronic apparatus1000, and the remaining one may be included in the server 2000. Also,model information established by the data learner 1310 may be providedto the data recognizer 1320 and data input to the data recognizer 1320may be provided as additional learning data to the data learner 1310 bywire or wirelessly.

At least one of the data learner 1310 and the data recognizer 1320 maybe implemented as a software module. When at least one of the datalearner 1310 and the data recognizer 1320 is implemented as a softwaremodule (or a program module including instructions), the software modulemay be stored in a non-transitory computer-readable recording medium.Also, in this case, at least one software module may be provided by anoperating system (OS) or a predetermined application. Alternatively, apart of at least one software module may be provided by an OS, and theremaining part may be provided by a predetermined application.

FIG. 30 is a block diagram of the data learner 1310 according to anembodiment of the disclosure.

Referring to FIG. 30, the data learner 1310 (e.g., model evaluator)according to an embodiment may include a data obtainer 1310-1, apre-processor 1310-2, a learning data selector 1310-3, a model learner1310-4, and a model evaluator 1310-5.

The data obtainer 1310-1 may obtain data needed to determine arecommendation situation. The data obtainer 1310-1 may obtain data(e.g., a photograph image) needed for learning to determine therecommendation situation. According to an embodiment, the model learner1310-4 may directly generate data needed to determine the recommendationsituation, or may receive data needed to determine the recommendationsituation from an external device or a server.

According to an embodiment, the data needed to determine therecommendation situation may include, but is not limited to, informationof a subject, surrounding environment information of the subject, andpersonalized learning data.

According to an embodiment, the data obtainer 1310-1 may obtain imagedata, voice data, text data, or bio-signal data. For example, the dataobtainer 1310-1 may receive data through an input device (e.g., amicrophone, a camera, or a sensor) of the electronic apparatus 1000.Alternatively, the data obtainer 1310-1 may obtain data through anexternal device that communicates with the electronic apparatus 1000.

The pre-processor 1310-2 may pre-process the obtained data so that theobtained data is used for learning for determining the recommendationsituation. The pre-processor 1310-2 may process the obtained data into apreset format so that the model learner 1310-4 that will be describedbelow may use the obtained data for learning for determining therecommendation situation.

For example, the pre-processor 1310-2 may generate one synthesized imageby overlapping at least some parts of a plurality of images based on acommon area included in a plurality of images (or frames) constitutingat least a part of an input video. In this case, a plurality ofsynthesized images may be generated from one video. The common area maybe an area including the same or similar common object (e.g., a solidobject, an animal/plant, or a person) in the plurality of images.Alternatively, the common area may be an area where colors, shades, RGBvalues, or CMYK values are the same or similar in the plurality ofimages.

The learning data selector 1310-3 may select data needed for learningfrom among pieces of pre-processed data. The selected data may beprovided to the model learner 1310-4. The learning data selector 1310-3may select the data needed for learning from among the pieces ofpre-processed data, according to a preset standard for determining therecommendation situation. Also, the learning data selector 1310-3 mayselect data according to a standard preset by learning by the modellearner 1310-4 that will be described below. For example, the learningdata selector 1310-3 may select image data including a photographingcomposition related to the information of the subject and thesurrounding environment information of the subject.

The model learner 1310-4 may learn a standard about how to determine therecommendation situation based on learning data. Also, the model learner1310-4 may learn a standard about which learning data is to be used inorder to determine the recommendation situation.

Also, the model learner 1310-4 may train a data recognition model usedto determine the recommendation situation by using the learning data. Inthis case, the data recognition model may be a model that ispre-established. For example, the data recognition model may be a modelthat is pre-established by receiving basic learning data (e.g., sampledata).

The data recognition model may be established in consideration of afield to which a recognition model is applied, the purpose of learning,or the computer performance of the electronic apparatus 1000. The datarecognition model may be a model based on, for example, a neuralnetwork. For example, a model such as a deep neural network (DNN), arecurrent neural network (RNN), or a bidirectional recurrent deep neuralnetwork (BRDNN) may be used as the data recognition model.

According to various embodiments, when a plurality of data recognitionmodels that are pre-established exist, the model learner 1310-4 maydetermine a data recognition model having a high relationship betweeninput learning data and basic learning data as the data recognitionmodel to be trained. In this case, the basic learning data may bepre-classified according to types of data, and the data recognitionmodel may be pre-established according to the types of data. Forexample, the basic learning data may be pre-classified according tovarious standards such as an area where the learning data is generated,a time for which the learning data is generated, a size of the learningdata, a genre of the learning data, a generator of the learning data,and a type of the subject in the learning data.

Also, the model learner 1310-4 may train the data recognition model byusing a learning algorithm including, for example, errorback-propagation or gradient descent.

Also, the model learner 1310-4 may train the data recognition modelthrough supervised learning by using, for example, the learning data asan input value. Also, the model learner 1310-4 may train the datarecognition model through unsupervised learning to find a standard fordetermining a situation by learning a type of data needed to determinethe situation by itself without supervision. Also, the model learner1310-4 may train the data recognition model through reinforcementlearning using a feedback about whether a result of determining thesituation according to learning is right.

Also, when the data recognition model is trained, the model learner1310-4 may store the trained data recognition model. In this case, themodel learner 1310-4 may store the trained data recognition model in thememory 1700 of the electronic apparatus 1000 including the datarecognizer 1320. Alternatively, the model learner 1310-4 may store thetrained data recognition model in the memory 1700 of the electronicapparatus 1000 including the data recognizer 1320. Alternatively, themodel learner 1310-4 may store the trained data recognition model in amemory of the server 2000 connected to the electronic apparatus 1000through a wired or wireless network.

In this case, the memory in which the trained data recognition model isstored may also store, for example, a command or data related to atleast another element of the electronic apparatus 1000. Also, the memorymay store software and/or a program. The program may include, forexample, a kernel, middleware, an application programming interface(API), and/or an application program (or an “application”).

When the model evaluator 1310-5 inputs evaluation data to the datarecognition model and a recognition result output from the evaluationdata does not satisfy a predetermined standard, the model evaluator1310-5 may cause the model learner 1310-4 to learn again. In this case,the evaluation data may be preset data for evaluating the datarecognition model.

For example, from among recognition results of the trained datarecognition model output from evaluation data, when the number or aratio of recognition results that are not accurate exceeds a presetthreshold value, it may be evaluated that the predetermined standard isnot satisfied. For example, when 2% is defined as the predeterminedstandard and wrong recognition results are output from more than 20pieces of evaluation data from among 1000 pieces of evaluation data, themodel evaluator 1310-5 may evaluate that the trained data recognitionmodel is not suitable.

When a plurality of trained data recognition models exist, the modelevaluator 1310-5 may evaluate whether each of the trained recognitionmodels satisfies a predetermined standard, and may determine a modelsatisfying the predetermined standard as a final data recognition model.In this case, when a plurality of models satisfy the predeterminedstandard, the model evaluator 1310-5 may determine one or apredetermined number of models that are preset in a descending order ofevaluation scores as final data recognition models.

At least one of the data obtainer 1310-1, the pre-processor 1310-2, thelearning data selector 1310-3, the model learner 1310-4, and the modelevaluator 1310-5 in the data learner 1310 may be manufactured as atleast one hardware chip and may be mounted on the electronic apparatus1000. For example, at least one of the model learner 1310-4, thepre-processor 1310-2, the learning data selector 1310-3, the modellearner 1310-4, and the model evaluator 1310-5 may be manufactured as adedicated hardware chip for AI, or may be manufactured as a part of anexisting general-purpose processor (e.g., a CPU or an applicationprocessor) or a graphics processor (e.g., a GPU) and may be mounted onthe electronic apparatus 1000.

Also, the data obtainer 1310-1, the pre-processor 1310-2, the learningdata selector 1310-3, the model learner 1310-4, and the model evaluator1310-5 may be mounted on one electronic apparatus 1000, or may beseparately respectively mounted on electronic apparatuses. For example,some of the data obtainer 1310-1, the pre-processor 1310-2, the learningdata selector 1310-3, the model learner 1310-4, and the model evaluator1310-5 may be included in the electronic apparatus 1000, and theremaining ones may be included in the server 2000.

Also, at least one of the data obtainer 1310-1, the pre-processor1310-2, the learning data selector 1310-3, the model learner 1310-4, andthe model evaluator 1310-5 may be implemented as a software module. Whenat least one of the data obtainer 1310-1, the pre-processor 1310-2, thelearning data selector 1310-3, the model learner 1310-4, and the modelevaluator 1310-5 is implemented as a software module (or a programmodule including instructions), the software module may be stored in anon-transitory computer-readable recording medium. Also, in this case,at least one software module may be provided by an OS or a predeterminedapplication. Alternatively, a part of at least one software module maybe provided by an OS, and the remaining part may be provided by apredetermined application.

FIG. 31 is a block diagram of the data recognizer 1320 according to anembodiment of the disclosure.

Referring to FIG. 31, the data recognizer 1320 according to anembodiment may include a data obtainer 1320-1, a pre-processor 1320-2, arecognition data selector 1320-3, a recognition result provider 1320-4,and a model refiner 1320-5.

The data obtainer 1320-1 may obtain data needed to determine arecommendation situation, and the pre-processor 1320-2 may pre-processthe obtained data so that the data obtained to determine a situation isused. The pre-processor 1320-2 may process the obtained data into apreset format so that the recognition result provider 1320-4 that willbe described below may use the data obtained to determine therecommendation situation.

The recognition data selector 1320-3 may select data needed to determinethe recommendation situation from among pieces of pre-processed data.The selected data may be provided to the recognition result provider1320-4. The recognition data selector 1320-3 may select some or all ofthe pieces of pre-processed data according to a preset standard fordetermining the recommendation situation. Also, the recognition dataselector 1320-3 may select data according to a standard preset bylearning by the model learner 1310-4 as described below.

The recognition result provider 1320-4 may determine the situation byapplying the selected data to a data recognition model. The recognitionresult provider 1320-4 may provide a recognition result according torecognition purpose of the data. The recognition result provider 1320-4may apply the selected data to the data recognition model by using thedata selected by the recognition data selector 1320-3 as an input value.Also, the recognition result may be determined by the data recognitionmodel.

For example, a recognition result of at least one image may be providedas text, a voice, a video, an image, or instructions (e.g., applicationexecution instructions or module function execution instructions). Forexample, the recognition result provider 1320-4 may provide arecognition result of an object included in the at least one image. Therecognition result may include, for example, pose information of theobject included in the at least one image, surrounding state informationof the object, and motion change information of the object included in avideo.

The model refiner 1320-5 may refine the data recognition model based onevaluation of the recognition result provided by the recognition resultprovider 1320-4. For example, the model refiner 1320-5 may provide therecognition result provided by the recognition result provider 1320-4 tothe model learner 1310-4 so that the model learner 1340-4 refines thedata recognition model.

At least one of the data obtainer 1320-1, the pre-processor 1320-2, therecognition data selector 1320-3, the recognition result provider1320-4, and the model refiner 1320-5 in the data recognizer 1320 may bemanufactured as at least one hardware chip and may be mounted on theelectronic apparatus 1000. For example, at least one of the dataobtainer 1320-1, the pre-processor 1320-2, the recognition data selector1320-3, the recognition result provider 1320-4, and the model refiner1320-5 may be manufactured as a dedicated hardware chip for AI, or maybe manufactured as a part of an existing general-purpose processor(e.g., a CPU or an application processor) or a graphics processor (e.g.,a GPU) and may be mounted on the electronic apparatus 1000.

Also, the data obtainer 1320-1, the pre-processor 1320-2, therecognition data selector 1320-3, the recognition result provider1320-4, and the model refiner 1320-5 may be mounted on one electronicapparatus 1000, or may be separately respectively mounted on electronicapparatuses. For example, some of the data obtainer 1320-1, thepre-processor 1320-2, the recognition data selector 1320-3, therecognition result provider 1320-4, and the model refiner 1320-5 may beincluded in the electronic apparatus 1000, and the remaining others maybe included in the server 2000.

Also, at least one of the data obtainer 1320-1, the pre-processor1320-2, the recognition data selector 1320-3, the recognition resultprovider 1320-4, and the model refiner 1320-5 may be implemented as asoftware module. When at least one of the data obtainer 1320-1, thepre-processor 1320-2, the recognition data selector 1320-3, therecognition result provider 1320-4, and the model refiner 1320-5 isimplemented as a software module (or a program module includinginstructions), the software module may be stored in a non-transitorycomputer-readable recording medium. Also, in this case, at least onesoftware module may be provided by an OS or a predetermined application.Alternatively, a part of at least one software module may be provided byan OS and the remaining part may be provided by a predeterminedapplication.

FIG. 32 is a block diagram illustrating an example where the electronicapparatus 1000 and the server 2000 interoperate to learn and recognizedata according to an embodiment of the disclosure. The server 2000according to an embodiment may include a data recognizer 2300. The datarecognizer 2300 may include a data obtainer 2310, a pre-processor 2320,a learning data selector 2330, a model learner 2340, and a modelevaluator 2350.

Referring to FIG. 32, the server 2000 may learn a standard fordetermining a recommendation situation, and the electronic apparatus1000 may determine the recommendation situation based on a learningresult of the server 2000.

In this case, a model learner 2340 of the server 2000 may perform afunction of the data learner 1310 of FIG. 29. The model learner 2340 ofthe server 2000 may learn a standard about which data is to be used inorder to determine the recommendation situation and how to determine therecommendation situation by using the data. The model learner 2340 mayobtain data to be used for learning, and may learn a standard fordetermining a situation by applying the obtained data to a datarecognition model that will be described below.

Also, the recognition result provider 1320-4 of the electronic apparatus1000 may determine the situation by applying data selected by therecognition data selector 1320-3 to the data recognition model generatedby the server 2000. For example, the recognition result provider 1320-4may transmit the data selected by the recognition data selector 1320-3to the server 2000, and the server 2000 may request to determine thesituation by applying the data selected by the recognition data selector1320-3 to a recognition model. Also, the recognition result provider1320-4 may receive information about information about the situationdetermined by the server 2000 from the server 2000.

Alternatively, the recognition result provider 1320-4 of the electronicapparatus 1000 may receive the recognition model generated by the server2000 from the server 2000, and may determine the recommendationsituation by using the received recognition model. In this case, therecognition result provider 1320-4 of the electronic apparatus 1000 maydetermine the recommendation situation by applying the data selected bythe recognition data selector 1320-3 to the data recognition modelreceived from the server 2000.

A method according to an embodiment may be embodied as program commandsexecutable by various computer means and may be recorded on acomputer-readable recording medium. The computer-readable recordingmedium may include program commands, data files, data structures, andthe like separately or in combinations. The program commands to berecorded on the computer-readable recording medium may be speciallydesigned and configured for embodiments of the disclosure or may bewell-known to and be usable by one of ordinary skill in the art ofcomputer software. Examples of the computer-readable recording mediuminclude a magnetic medium such as a hard disk, a floppy disk, or amagnetic tape, an optical medium such as a compact disk read-only memory(CD-ROM) or a digital versatile disk (DVD), a magneto-optical mediumsuch as a floptical disk, and a hardware device specially configured tostore and execute program commands such as a ROM, a RAM, or a flashmemory. Examples of the program commands are advanced language codesthat may be executed by a computer by using an interpreter or the likeas well as machine language codes made by a compiler.

Some embodiments may be implemented as a recording medium includingcomputer-readable instructions such as a computer-executable programmodule. The computer-readable medium may be an arbitrary availablemedium accessible by a computer, and examples thereof include allvolatile and non-volatile media and separable and non-separable media.Further, examples of the computer-readable medium may include a computerstorage medium and a communication medium. Examples of the computerstorage medium include all volatile and non-volatile media and separableand non-separable media, which are implemented by an arbitrary method ortechnology, for storing information such as computer-readableinstructions, data structures, program modules, or other data. Thecommunication medium typically includes computer-readable instructions,data structures, program modules, other data of a modulated data signal,or other transmission mechanisms, and examples thereof include anarbitrary information transmission medium. Also, some embodiments may beimplemented as a computer program or a computer program productincluding computer-executable instructions such as a computer programexecuted by a computer.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A method for photographing a subject using arecommended photographing composition by an electronic device includinga camera and a display, the method comprising: identifying the subjectin a preview image, obtained from the camera and displayed on thedisplay; obtaining information of the subject being tracked in thepreview image, the obtained information including at least one of anorientation or a position of the subject being tracked in the previewimage; obtaining, using an Artificial Intelligence (AI) model, therecommended photographing composition based on the information of thesubject; displaying an indicator guiding a user to move the electronicdevice such that the subject is at a position in the preview imagecorresponding to the recommended photographing composition; andphotographing the subject, based on the subject being at the position inthe preview image corresponding to the recommended photographingcomposition obtained using the AI model.
 2. The method of claim 1,wherein the recommended photographing composition is obtained basedfurther on a golden ratio composition information.
 3. The method ofclaim 1, wherein the recommended photographing composition is obtainedbased further on a centered composition information.
 4. The method ofclaim 1, wherein the indicator includes a shape corresponding to a shapeof at least part of a human body.
 5. The method of claim 1, furthercomprising: identifying a number of one or more subjects included in thepreview image, wherein the recommended photographing composition isobtained based further on the identified number of the one or moresubjects included in the preview image.
 6. The method of claim 5,wherein based on the number of the one or more subjects being identifiedas singular, obtaining the recommended photographing composition usingthe AI model.
 7. The method of claim 1, further comprising: presenting adirection indication for the movement of the electronic device accordingto the recommended photographing composition, wherein the directionindication is presented by at least one of a graphical indication, atext indication, or a speech indication.
 8. An electronic device forphotographing a subject using a recommended photographing composition,the electronic device comprising: a camera; a display; and at least oneprocessor configured to: identify the subject in a preview image,obtained from the camera and displayed on the display; obtaininformation of the subject being tracked in the preview image, theobtained information including at least one of an orientation or aposition of the subject being tracked in the preview image; obtain,using an Artificial Intelligence (AI) model, the recommendedphotographing composition based on the information of the subject;display an indicator guiding a user to move the electronic device suchthat the subject is at a position in the preview image corresponding tothe recommended photographing composition; and photograph the subject,based on the subject being at the position in the preview imagecorresponding to the recommended photographing composition obtainedusing the AI model.
 9. The electronic device of claim 8, wherein therecommended photographing composition is obtained based further on agolden ratio composition information.
 10. The electronic device of claim8, wherein the recommended photographing composition is obtained basedfurther on a centered composition information.
 11. The electronic deviceof claim 8, wherein the indicator includes a shape corresponding to ashape of at least part of a human body.
 12. The electronic device ofclaim 8, wherein the at least one processor is further configured to:identify a number of one or more subjects included in the preview image,and wherein the recommended photographing composition is obtained basedfurther on the identified number of the one or more subjects included inthe preview image.
 13. The electronic device of claim 12, wherein basedon the number of the one or more subjects being identified as singular,the recommended photographing composition is obtained using the AImodel.
 14. The electronic device of claim 8, wherein the at least oneprocessor is further configured to: present a direction indication forthe movement of the electronic device according to the recommendedphotographing composition, and wherein the direction indication ispresented by at least one of a graphical indication, a text indication,or a speech indication.
 15. The electronic device of claim 8, whereinthe subject in the preview image is tracked using the AI model.
 16. Theelectronic device of claim 8, wherein the subject is automaticallyphotographed based on the subject being at the position in the previewimage corresponding to the recommended photographing compositionobtained using the AI model.
 17. The electronic device of claim 8,wherein the AI model is trained by a plurality of images including aplurality of composition information for recommending a photographingcomposition.
 18. A non-transitory computer-readable recording mediumhaving recorded thereon a program, which when executed by at least oneprocessor of an electronic device including a camera and a display,causes the at least one processor to perform a method for photographinga subject using a recommended photographing composition, the methodcomprising: identifying the subject in a preview image, obtained fromthe camera and displayed on the display; obtaining information of thesubject being tracked in the preview image, the obtained informationincluding at least one of an orientation or a position of the subjectbeing tracked in the preview image; obtaining, using an ArtificialIntelligence (AI) model, the recommended photographing composition basedon the information of the subject; displaying an indicator guiding auser to move the electronic device such that the subject is at aposition in the preview image corresponding to the recommendedphotographing composition; and photographing the subject, based on thesubject being at the position in the preview image corresponding to therecommended photographing composition obtained using the AI model. 19.The non-transitory computer-readable recording medium of claim 18,wherein the recommended photographing composition is obtained basedfurther on a golden ratio composition information.
 20. Thenon-transitory computer-readable recording medium of claim 18, whereinthe recommended photographing composition is obtained based further on acentered composition information.
 21. The non-transitorycomputer-readable recording medium of claim 18, wherein the firstindicator includes a shape corresponding to a shape of at least part ofa human body.
 22. The non-transitory computer-readable recording mediumof claim 18, wherein the method further comprises: identifying a numberof one or more subjects included in the preview image, and wherein therecommended photographing composition is obtained based further on theidentified number of the one or more subjects included in the previewimage.
 23. The non-transitory computer-readable recording medium ofclaim 22, wherein based on the number of the one or more subjects beingidentified as singular, obtaining the recommended photographingcomposition by using the AI model.
 24. The non-transitorycomputer-readable recording medium of claim 18, wherein the methodfurther comprises: presenting a direction indication for the movement ofthe electronic device according to the recommended photographingcomposition, and wherein the direction indication is presented by atleast one of a graphical indication, a text indication, or a speechindication.
 25. The non-transitory computer-readable recording medium ofclaim 18, wherein the subject in the preview image is tracked using theAI model.
 26. The non-transitory computer-readable recording medium ofclaim 18, wherein the subject is automatically photographed based on thesubject being at the position in the preview image corresponding to therecommended photographing composition obtained using the AI model. 27.The non-transitory computer-readable recording medium of claim 18,wherein the AI model is trained by a plurality of images including aplurality of composition information for recommending a photographingcomposition.
 28. The non-transitory computer-readable recording mediumof claim 18, wherein the method further comprises: maintaining aposition of the indicator in the preview image as the preview imagechanges due to the movement of the electronic device, and wherein adistance between the indicator and the subject in the preview imagechanges as the preview image changes due to the movement of theelectronic device.
 29. The non-transitory computer-readable recordingmedium of claim 18, wherein the method further comprises: discontinuingthe display of the indicator, based on the subject being at the positionin the preview image corresponding to the recommended photographingcomposition obtained using the AI model.
 30. The non-transitorycomputer-readable recording medium of claim 18, wherein the methodfurther comprises: visually indicating that the subject is at theposition in the preview image corresponding to the recommendedphotographing composition obtained using the AI model.